Providing an augmented reality overlay for display over a view of a user

ABSTRACT

The present disclosure is directed toward systems and methods for generating and providing an augmented reality overlay for display in connection with an augmented reality display device. For example, systems and methods described herein identify a user being viewed through an augmented reality display device, and builds an augmented reality overlay for the user that is displayed on a view of the user through the augmented reality display device. Systems and methods described herein build the augmented reality overlay based on the location of the augmented reality display device, and on other networking system information including a networking system relationship between the user wearing the augmented reality display device and the user who is being looked at through the augmented reality display device.

BACKGROUND

Augmented reality provides users with views of physical, real-worldenvironments that have been augmented by sound, video, and/or graphics.Augmented reality systems and applications provide a number ofadvantages in technology areas, such as entertainment and gaming. Toillustrate, augmented reality games, such as those playable using mobiledevices (e.g., smartphones), are becoming increasingly commonplace andpopular. However, the boundaries of augmented reality have just begun tobe explored. In particular, conventional augmented reality systems havemerely scratched the surface with regard to the application of augmentedreality in the day-to-day activities of user. Thus, there are a numberof disadvantages corresponding to conventional augmented realitysystems.

SUMMARY

One or more embodiments described herein provide benefits and/or solveone or more of the foregoing or other problems in the art with augmentedreality systems. For example, the present application discloses variousembodiments of systems, methods, and computer-readable media thatprovide an augmented reality framework that alters the way a networkingsystem user looks and speaks. The systems and methods described hereinutilize information associated with a user (e.g., as collected by asocial networking system) to generate an augmented reality overlay forthe user to alter the user's appearance. To illustrate, the augmentedreality overlay can include attire, jewelry, makeup, hairstyle choices,etc. that are appropriate for the user's circumstances. The systems andmethods described herein then anchor the augmented reality overlay tothe user within a display of an augmented reality device, such that theuser appears to be wearing the attire, along with the makeup, hairstyle,etc. that are included in the augmented reality overlay.

Additionally, in at least one embodiment, an augmented reality overlayis customized based on the identity of a viewing user. In other words,each viewing user can be provided a unique view of the user based on avariety of factors and other information associated with a relationshipbetween the viewing user and the user being viewed. For example, thedisclosed augmented reality system can utilize networking systemrelationship information to determine that a first attire choice for theuser is appropriate to display to the user's employer, while a secondattire choice for the user is appropriate to display to the user'sfriend. Thus, the disclosed augmented reality system takes an additionallayer of social factors into account beyond simple locationconsiderations when generating a user's augmented reality overlay.

Furthermore, the disclosed augmented reality system includes an auditoryelement that enables seamless and instantaneous speech translationwithin the augmented reality framework. For example, one or moreembodiments include translation services that enable a first user and asecond user to hold a normal conversation, even though both users speakdifferent languages. In at least one embodiment, systems and methodsdescribed herein include a graphical augmented reality element thatmakes a user's mouth appear to be synchronized to machine-translatedspeech. Thus, systems and methods described herein provide theappearance that the user is speaking the language of the listening user,regardless of the actual language being spoken by the speaking user.

The following description sets forth additional features and advantagesof one or more embodiments of the disclosed systems and methods. In somecases, such features and advantages will be obvious to a skilled artisanfrom the description or may be learned by the practice of the disclosedembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

The detailed description refers to the drawings briefly described below.

FIG. 1 illustrates an environmental diagram of a augmented realitysystem in accordance with one or more embodiments;

FIGS. 2A-2D illustrate a series of views illustrating various featuresin accordance with one or more embodiments;

FIGS. 3A-3C illustrate a series of views illustrating various featuresin accordance with one or more embodiments;

FIG. 4 illustrates a detailed schematic diagram of the augmented realitysystem in accordance with one or more embodiments;

FIG. 5 illustrates a flowchart of a series of acts in a method ofdisplaying an augmented reality overlay on a view of a user inaccordance with one or more embodiments;

FIG. 6 illustrates a block diagram of an exemplary computing device inaccordance with one or more embodiments;

FIG. 7 is an example network environment of a networking system inaccordance with one or more embodiments; and

FIG. 8 illustrates a social graph in accordance with one or moreembodiments.

DETAILED DESCRIPTION

One or more embodiments described herein provide benefits and/or solveone or more of the foregoing or other problems in the art with systemsand methods for building an augmented reality framework that enables anetworking system user to appear and sound appropriately regardless ofhis location and language skills. For example, utilizing the augmentedreality system described herein, a networking system user can attend aformal event in a different country and appear to other users to bewearing expensive evening attire appropriate for the country andspeaking the language of the country, even though the user is actuallywearing informal clothes and cannot speak the country's language. Thus,the augmented reality system enables users to pare down their wardrobes,travel with confidence, and move from one event to the next withoutneeding to change.

As used herein, “augmented reality” refers to a technology that createsa composite view for a user including computer-generated elements inassociation with the user's real-life view. For example, in one or moreembodiments, the augmented reality system generates an augmented realityoverlay including one or more augmented reality elements, and anchorsthat augmented reality overlay over the user's view within an augmentedreality display device. As used herein, an “augmented reality overlay”refers to a graphical image that overlays part or all of a user withinan augmented reality view of the user. In one or more embodiments, anaugmented reality overlay is composed of one or more augmented realityelements.

As used herein, an “augmented reality element” refers tocomputer-generated elements that comprise part of an augmented realityoverlay. An augmented reality element, as used herein, can be one ofmany types. For example, an augmented reality element can be a displayor graphical element (e.g., representing an article of clothing) or canbe an effect (e.g., such as changing eye color, removing facial hair,etc.). In one example, an augmented reality overlay includes augmentedreality elements such as a tuxedo, an expensive watch, a fresh haircut,and trimmed facial hair. In yet further embodiments, an augmentedreality element can include any other computer-generated visual and/oraudio content for augmenting a view of a real-world environment.

In one or more embodiments, the augmented reality system describedherein requires users to utilize augmented reality devices. For example,an augmented reality device can be a head-mounted display device (e.g.,GOOGLE GLASS®, MICROSOFT HOLOLENS®, etc.) such that a user's trueappearance is effectively masked behind a displayed augmented realityoverlay. In at least one embodiment, the augmented reality displaydevice also includes audio features (e.g., headphones, ear buds, etc.)that provide audio playback to the user wearing the device.

In one illustrative example of the augmented reality system, theaugmented reality system initially enables a user to define augmentedreality rules or preferences via a networking system. For example,augmented reality system users can configure rules that establish attireguidelines (e.g., “Never put me in clothing that is orange,” “Never showme wearing a dress,” etc.), and other appearance choices (e.g., “Ialways want to be shown with facial hair,” “Don't change my hairstyle,”etc.). In one or more embodiments, the augmented reality systemautomatically generates a augmented reality overlay for a user, butrelies on the user's pre-established rules or preferences as boundarieswithin which to operate. In other words, the user's pre-establishedpreferences serve to keep the augmented reality system from generating aaugmented reality overlay that includes augmented reality elements thatthe user would dislike.

As mentioned above, in one or more embodiments, the augmented realitysystem assumes that all users are constantly utilizing an augmentedreality display device (e.g., a head-mounted display device or“augmented reality display device”). In at least one embodiment, inorder for the augmented reality system to effectively ensure allaugmented reality system users look and sound appropriately for a givensetting, all users must be utilizing an augmented reality displaydevice. Thus, all augmented reality system users can then view augmentedreality overlays and hear translated speech. Accordingly, theembodiments described herein assume that all augmented reality systemusers are utilizing an augmented reality display device.

In at least one embodiment, the augmented reality system firstdetermines a user's location and setting in order to determine thecorrect appearance and language options available for generating anaugmented reality overlay. For example, when an augmented reality systemuser walks into a formal dinner event at an upscale restaurant, theaugmented reality system detects the user's location and setting inorder to determine that formal attire is appropriate. In one or moreembodiments, the augmented reality system detects the user's locationutilizing a GPS signal and/or a WiFi signal from the user's augmentedreality display device. Furthermore, the augmented reality systemdetermines the user's setting by performing image analysis on an imagecaptured by the user's augmented reality display device.

Next, the augmented reality system identifies the co-user at whom theuser is looking. As discussed above, the augmented reality systemprovides an augmented reality overlay for an augmented reality systemco-user who is being looked at through an augmented reality displaydevice of an augmented reality system user. Thus, the co-user does notsee the augmented reality overlay provided by the augmented realitysystem to the user, even though the augmented reality overlay isgenerated in association with a view of the co-user. Accordingly, whenthe user first looks at the co-user, the augmented reality systemidentifies the co-user in order to determine whether the co-user is anaugmented reality system user. In one or more embodiments, the augmentedreality system utilizes a video stream of images captured by a cameraassociated with the augmented reality display device of the user inorder to identify the co-user. For example, the augmented reality systemcan utilize facial recognition techniques, image matching, eye scanning,and so forth in order to identify the co-user and to determine that theco-user is an augmented reality user.

After identifying the co-user, the augmented reality system identifiesthe co-user's pre-configured rules or preferences, and begins buildingan augmented reality overlay for the co-user. In at least oneembodiment, the augmented reality system begins building the co-user'saugmented reality overlay by performing a 3D scan on an image of theco-user from the video stream captured by the user's augmented realitydisplay device. For example, the 3D scan of the image determines the3-dimensional shape of the co-user by creating a point cloud of datafrom the co-user's 2-dimensional surface, as shown in the capturedimage. In one or more embodiments, the augmented reality system thenbuilds a graphical mesh on the co-user's 3-dimensional shape. In atleast one embodiment, the augmented reality system can attach one ormore augmented reality elements to the graphical mesh in order togenerate a augmented reality overlay for the co-user.

Accordingly, once the augmented reality system has built the graphicalmesh representing the co-user's 3-dimensional shape, the augmentedreality system can determine appropriate augmented reality elements toattach to the graphical mesh. In one or more embodiments, the augmentedreality system determines appropriate augmented reality elements byselecting augmented reality elements that are appropriate to thelocation and setting of the user and the co-user, and scoring theselected elements based on a variety of other considerations. Theaugmented reality system then attaches the highest scoring augmentedreality elements to the graphical mesh.

For example, if the user and the co-user are located at an upscalerestaurant in the evening, the augmented reality system will select acollection of augmented reality elements that are appropriate for thatsetting (e.g., evening wear, high-end jewelry, modern hairstyles, etc.).Following this, the augmented reality system calculates a score for eachelement in the collection of augmented reality elements based on avariety of factors.

In at least one embodiment, the augmented reality system calculates ascore for an augmented reality element based on the co-user'spre-configured rules or preferences. For example, if the co-userpreviously configured a set of rules that indicate he never wants to beshown wearing coat-tails or a top hat, the augmented reality system willcalculate a low score for augmented reality elements that include suitswith coat-tails or hats that are top-hats. Similarly, if the co-userpreviously configured a set of rules that indicated he prefers to beshown wearing the color black with his hair parted on the right side,the augmented reality system will calculate a high score for augmentedreality elements that include black attire or hairstyles with aright-hand part.

After scoring each of the collection of augmented reality elements basedon the co-user's pre-configured rules or preferences, the augmentedreality system can further weight the scores calculated for eachaugmented reality element based on networking system information. Forexample, in one or more embodiments, the augmented reality systemfunctions as part of a networking system, and the user and co-user arenetworking system users. Accordingly, in at least one embodiment, theaugmented reality system can gather data about the location of the userand co-user from the networking system. To illustrate, the networkingsystem may have additional information about the formal event that bothusers are attending (e.g., a requested dress code, an itinerary for theevent, etc.). Additionally, the networking system may have additionalrelevant information, such as the weather conditions. Thus, theaugmented reality system can further calculate the score for each of thecollection of augmented reality elements based on this additionalnetworking system data.

Furthermore, the augmented reality system can weight the scorescalculated for each augmented reality element based on a relationshipcoefficient between the user and the co-user. In one or moreembodiments, the augmented reality system calculates a relationshipcoefficient that numerically represents the strength of the relationshipbetween two users. For example, the relationship coefficient is informedby an amount of networking system activity common to both users (e.g.,sharing each other's posts, tagging each other in posts, checking in atthe same locations, etc.), in addition to their profile information(e.g., indicating they work for the same company, live in the same area,have the same hobbies, are related, have been networking system“friends” for a threshold period of time, etc.). Accordingly, theaugmented reality system can score certain augmented reality elementshigher when the relationship coefficient between the user and theco-user is high, and vice versa.

For instance, the user and the co-user may attend the formal eventdescribed above and share a high relationship coefficient. Theirrelationship coefficient may be high because they frequently tag eachother in posts related to a particular sports team, and have checked-inat that sports team's games together in the past. In light of this, theaugmented reality system may score certain augmented reality elementshigher. For example, the augmented reality system may score an augmentedreality element displaying a neck tie in the color of the sports teamhigher, even though it is a fairly unusual color.

After the augmented reality system has calculated a score for each ofthe collection of augmented reality elements, the augmented realitysystem selects the highest scoring augmented reality elements and startsbuilding the co-user's augmented reality overlay. In at least oneembodiment, the augmented reality system builds the co-user's augmentedreality overlay by attaching the highest scoring augmented realityelements to the graphical mesh representing the co-user's 3-dimensionalshape. For example, in attaching an augmented reality elementrepresenting a black tuxedo to the co-user's graphical mesh, theaugmented reality system can resize the augmented reality element inorder to cover the appropriate areas of the graphical mesh (e.g.,lengthen sleeves, shorten trousers, widen a waistband, etc.). In one ormore embodiments, the augmented reality system attaches the highestscoring augmented reality elements to the graphical mesh according to aset of reality-based rules (e.g., a person should not wear more than onepair of pants, a person should not wear sunglasses indoors or at night,etc.).

Once the augmented reality system attaches one or more augmented realityelements to the co-user's graphical mesh, the graphical mesh becomes theco-user's augmented reality overlay. At this point, the augmentedreality system provides the co-user's augmented reality overlay to theaugmented reality display device of the user. In order to correctlydisplay the co-user's augmented reality overlay, the augmented realitysystem anchors the augmented reality overlay within the user's augmentedreality display device. For example, in order to account for themovements of both the user and the co-user (e.g., shifting from one foodto the other, head tilts, etc.), the augmented reality system anchorsthe augmented reality overlay within the user's view via the augmentedreality display device. At this point, the co-user appears to the userto be wearing different attire than what he is really wearing, and theco-user's actual attire is successfully masked from the user via theuser's augmented reality display device.

In one or more embodiments, the augmented reality system not onlyensures that a user is appropriately attired for a particular setting,the augmented reality system also ensures that the user can effectivelycommunicate in the particular setting. For example, as mentioned above,an augmented reality display device can include some type of headphonethat enables the user to hear auditory playback. Thus, utilizingnetworking system information as well as location information, theaugmented reality system can determine what language is being spoken toa user and what language(s) the user is capable of understanding. Theaugmented reality system can then provide translation services when thelanguage being spoken to a user is not a language that the user canunderstand.

Furthermore, in at least one embodiment, the augmented reality systemcan provide an augmented reality element within an augmented realityoverlay that makes the user's mouth appear to be synchronized with anymachine-translation provided by the augmented reality system. Toillustrate, if the co-user is speaking French and the user onlyunderstands English, the augmented reality system translates theco-user's French into English, provides the English translation to theuser via the second user's augmented reality display device headphones,and adds an augmented reality element to the co-user's augmented realityoverlay that displays a digital mouth overlaid on the co-user's mouththat appears to be synchronized to the English translation. Theaugmented reality system can provide these same features with regard tothe user such that the co-user can understand what the user says inEnglish.

FIG. 1 illustrates an example block diagram of an environment forimplementing the augmented reality system 100. As illustrated in FIG. 1,the augmented reality system 100 includes the augmented reality displaydevices 102 a, 102 b, the server device(s) 106, and the augmentedreality marketplace 114, which are communicatively coupled through anetwork 112. As shown in FIG. 1, the augmented reality display devices102 a, 102 b include the networking system application 104 a, 104 b,respectively. Additionally shown in FIG. 1, the server device(s) 106includes a networking system 108. Further shown in FIG. 1, the augmentedreality display devices 102 a, 102 b are associated with the users 110a, 110 b, respectively.

The augmented reality display devices 102 a, 102 b, the server device(s)106, and the augmented reality marketplace 114 communicate via thenetwork 112, which may include one or more networks and may use one ormore communication platforms or technologies suitable for transmittingdata and/or communication signals. In one or more embodiments, thenetwork 112 includes the Internet or World Wide Web. The network 112,however, can include various other types of networks that use variouscommunication technologies and protocols, such as a corporate intranet,a virtual private network (“VPN”), a local area network (“LAN”), awireless local network (“WLAN”), a cellular network, a wide area network(“WAN”), a metropolitan area network (“MAN”), or a combination of two ormore such networks. Although FIG. 1 illustrates a particular arrangementof augmented reality display devices 102 a, 102 b, the server device(s)106, the augmented reality marketplace 114, and the network 112, variousadditional arrangements are possible. For example, the augmented realitydisplay devices 102 a, 102 b may directly communicate with thenetworking system 108, bypassing the network 112. Further, the augmentedreality system 100 can include any number of augmented reality displaydevices communicating with the networking system 108. Additional detailsrelating to the network 112 are explained below with reference to FIG.7.

In one or more embodiments, the augmented reality marketplace 114 is anexternal server wherein augmented reality system users can obtainadditional augmented reality elements for inclusion in one or moreaugmented reality overlays. For example, as will be discussed furtherbelow, an augmented reality overlay is composed of one or more augmentedreality elements. In at least one embodiment, a user can specify thataugmented reality system 100 includes certain augmented reality elementsin all augmented reality overlays associated with the user. Inadditional or alternative embodiments, the augmented reality marketplace114 is integrated with and hosted by the networking system 108.

To illustrate, other entities (e.g., businesses, celebrities, artists,etc.) can generate augmented reality elements for sale via the augmentedreality system 100. For example, a famous artist may generate a limitededition augmented reality element that represents a piece of jewelry. Ifan augmented reality system user purchases an augmented reality elementvia the augmented reality marketplace 114, the user may specify that theaugmented reality system 100 include the purchased augmented realityelement in future augmented reality overlays associated with the user.Additionally or alternatively, the user can define rules that specifywhen the augmented reality system 100 should include the purchasedaugmented reality element in an augmented reality overlay (e.g., only onformal occasions, only on certain holidays, etc.).

As mentioned above, the augmented reality display devices 102 a, 102 binclude an augmented reality display, a video capturing device (e.g., adigital camera), and an audio playback mechanism (e.g., headphones). Forexample, in one or more embodiments, the augmented reality display ofthe augmented reality display devices 102 a, 102 b displays a augmentedreality overlay displayed in connection with on the wearer's normalview. In at least one embodiment, the augmented reality display operatesas a pair of lenses (e.g., eye glass lenses, contact lenses, etc.)positioned over the wearer's eyes. Additionally, in one or moreembodiments, the video capturing devices associated with the augmentedreality display devices 102 a, 102 b are micro digital video camerasmounted (e.g., to an ear piece, or over the bridge of the wearer's nose)to the augmented reality display devices 102 a, 102 b, respectively.Further, the audio playback mechanism of the augmented reality displaydevices 102 a, 102 b may include headphones, ear buds, or speakers builtinto a portion of the augmented reality display devices 102 a, 102 b(e.g., built into the ear pieces). Thus, in a preferred embodiment, theaugmented reality display devices 102 a, 102 b are similar to eyeglasses with all the component parts built-in. In one or moreembodiments, the augmented reality display devices 102 a, 102 b alsoinclude a processor capable of executing software code.

In at least one embodiment, the users 110 a, 110 b of the augmentedreality display devices 102 a, 102 b, respectively, are users of thenetworking system 108. For example, in at least one embodiment, theusers 110 a, 110 b are networking system users and can engage innetworking system activities (e.g., liking and sharing posts, uploadingand sharing digital pictures and videos, checking in at variouslocations, etc.). In one embodiment, the users 110 a, 110 b are also“friends” (e.g., associated co-users) via the networking system 108.Further, in at least one embodiment described herein, the user 110 a isthe augmented reality system user looking at the user 110 b (e.g., theco-user). Thus, in that embodiment, the augmented reality system 100provides an augmented reality overlay to the augmented reality displaydevice 102 a that is anchored over the user's 110 a view of the user 110b.

For example, as illustrated in the overview shown in FIG. 2A, the user110 a and the user 110 b are both wearing the augmented reality displaydevices 102 a and 102 b, respectively. In order to illustrate thefeatures of the augmented reality system 100, one or more embodimentsdescribed herein are described as the user 110 a looking at the user 110b. Thus, for purposes of these embodiments, the user 110 b may bereferred to as the co-user of the user 110 a. In other words, for thepurposes of the description herein, the “co-user” is the one for whomthe augmented reality system 100 is generating an augmented realityoverlay, and the “user” is the one to whom the augmented reality system100 provides the generated augmented reality overlay. Thus, in FIGS.2A-2D, the “user” is the user 110 a, and the “co-user” is the user 110b.

For instance, FIG. 2B illustrates an image 202 of the user 110 bcaptured by the augmented reality display device 102 a associated withthe user 110 a. In other words, the image 202 presents the same viewthat the user 110 a would see the user 110 b through the augmentedreality display device 102 a, prior to activating the augmented realitysystem 100. In other words, the view 204 is of the user 110 b with noadditional augmented reality overlay provided by the augmented realitysystem 100. As shown in FIG. 2B, the user 110 b, in real life, iswearing jeans, a t-shirt, and the augmented reality device 102 b. In oneor more embodiments and in response to the user 110 a activating theaugmented reality system 100, the augmented reality display device 102 aprovides the image 202 to the augmented reality system 100.

As mentioned above, the user 110 a activates the augmented realitysystem 100 in order to view an augmented reality overlay in connectionwith the user 110 b. In one or more embodiments, the augmented realitysystem 100 activates in response to the user 110 a powering on theaugmented reality display device 102 a. Additionally or alternatively,the augmented reality system 100 may activate in response to determiningthat the user 110 a is looking at another person (e.g., by monitoringthe eyes of the user 110 a, by performing image analysis in connectionwith an image captured by the augmented reality display device 102 a,etc.).

Upon activating, the augmented reality system 100 first determineswhether the user 110 b is an augmented reality system user. In one ormore embodiments, the augmented reality system 100 determines whetherthe user 110 b is an augmented reality system user by determining theidentity of the user 110 b. For example, in at least one embodiment, theaugmented reality display device 102 a provides at least one capturedimage including the user 110 b (e.g., the image 202) to the augmentedreality system 100. The augmented reality system 100 then utilizesfacial recognition techniques, image matching, eye scanning, etc. inconnection with the provided image in order to identify the user 110 b.For instance, the augmented reality system 100 can utilize facialrecognition techniques to match the image of the user 110 b to anexisting profile picture within the networking system 108.

In one or more embodiments, in response to determining the user 110 b isan augmented reality system user, the augmented reality system 100performs a threshold determination as to the initial appearance of theuser 110 b. For example, in some instances, the user 110 b may beappropriately attired for the location and setting in which theaugmented reality system 100 activated. In that case, the augmentedreality system 100 may determine to exit the process of generating anaugmented reality system overlay for the user 110 b. Alternatively, ifthe user 110 b is wearing appropriate attire, the augmented realitysystem 100 may generate a partial augmented reality system overlay thatincludes other augmented reality elements other than clothing elements.In one or more embodiments, the augmented reality system 100 performsthis threshold determination using computer vision techniques, imagecomparisons, Internet searches, and other machine learning techniques.

In at least one embodiment, the augmented reality system 100 determinesthat the user 110 b is not a user of the augmented reality system 100.For example, the user 110 b is not a networking system user, and/or hasnot opted into the features of the augmented reality system 100. If theuser 110 b is not a user of the augmented reality system 100, theaugmented reality system 100 may provide no augmented reality overlayfor display in association with a view of the user 110 b. Alternatively,the augmented reality system 100 may provide a default augmented realityoverlay that is appropriate to the location and setting of the user 110a. In one or more embodiments, the default augmented reality overlay mayinclude simple attire with no extra enhancements.

In response to determining the identified user 110 b is a user of theaugmented reality system 100 (e.g., the user 110 b has a networkingsystem account and has opted in to the features of the augmented realitysystem 100), the augmented reality system 100 performs athree-dimensional scan of the user 110 b displayed the image provided bythe augmented reality display device 102 a. As mentioned above, theaugmented reality system 100 utilizes the three-dimensional scan of theuser 110 b in order to build a graphical mesh that represents thethree-dimensional shape of the user 110 b. Later, the augmented realitysystem 100 will utilize the graphical mesh to generate an augmentedreality overlay for the user 110 b.

In one or more embodiments, the augmented reality system 100 nextdetermines the location and setting of the augmented reality displaydevices 102 a and 102 b. As mentioned above, in one or more embodiments,the augmented reality display devices 102 a, 102 b provide locationinformation to the augmented reality system 100 including, but notlimited to, GPS data, WiFi data, biometric data (e.g., a step count),and so forth. Utilizing this location information in combination withmap data, networking system information, Internet searches, and soforth, the augmented reality system 100 determines the location of theaugmented reality display devices 102 a, 102 b.

In at least one embodiment, the augmented reality system 100 calculatesa confidence score related to the determined location of the augmentedreality display devices 102 a, 102 b. For example, in one embodiment,the augmented reality display devices 102 a, 102 b are located on acertain floor of a building with many floors. Thus, it may be difficultfor the augmented reality system 100 to determine, based on GPS data,exactly where in the building the augmented reality display devices 102a, 102 b are located. Accordingly, the augmented reality system 100 canutilize other types of location information (e.g., networking systemevent information that the user 110 a has saved within the networkingsystem 108 for the current date and time) to fine-tune the determinedlocation of the augmented reality display devices 102 a, 102 b. As theaugmented reality system 100 supplements GPS data with other types oflocation information, the augmented reality system 100 can determine alevel of confidence, or confidence score, that indicates how likely thedetermined location of the augmented reality display devices 102 a, 102b is correct. If the augmented reality system 100 calculates aconfidence score that is below a threshold amount, the augmented realitysystem 100 can prompt the user 110 a to confirm his location, or toprovide additional location information.

The augmented reality system 100 can further determine the settingassociated with the augmented reality display devices 102 a, 102 b. Forexample, in one or more embodiments, the augmented reality system 100utilizes image analysis in connection with the image captured by theaugmented reality display device 102 a to determine whether theaugmented reality display devices 102 a, 102 b are located in a formalor informal setting, in a business or casual setting, in a public orprivate setting, and so forth. In determining the setting associatedwith the augmented reality display devices 102 a, 102 b, the augmentedreality system 100 further informs the appropriate selection ofaugmented reality elements for inclusion in the augmented realityoverlay, as will be described further below. The augmented realitysystem 100 can further utilize networking system information, Internetsearches, and so forth in determining the setting within a predetermineddegree of confidence, as described above.

To illustrate the location and setting determination performed by theaugmented reality system 100, FIG. 2B shows an image 202 captured by theaugmented reality display device 102 a. In other words, the image 202includes what the user 110 a is seeing via the augmented reality displaydevice 102 a. Accordingly, the augmented reality display device 102 aprovides the image 202 to the augmented reality system 100, along withother location information as described above. As further describedabove, the augmented reality system 100 performs object analysis toidentify a conference room table and chairs, a white board, and soforth. From these details, in combination with the other locationinformation provided by the augmented reality display device 102 a, theaugmented reality system 100 determines that the augmented realitydisplay devices 102 a, 102 b are located in the board room of a company,and that the users 110 a, 110 b are attending a board meeting.

In response to determining the location and setting of the augmentedreality display devices 102 a, 102 b and that the user 110 b is anaugmented reality system user, the augmented reality system 100 nextidentifies a collection of augmented reality elements that areappropriate to that location and setting. As described above, theaugmented reality system 100 begins the process of generating anaugmented reality overlay by identifying appropriate augmented realityelements relative to the location and setting of the augmented realitydisplay devices 102 a, 102 b. Accordingly, in response to determiningthat the augmented reality display devices 102 a, 102 b are located at abusiness and that the users 110 a, 110 b are attending a board meeting,the augmented reality system 100 can collect appropriate augmentedreality elements. For example, the augmented reality system 100 maycollect augmented reality elements representing business attire (e.g.,suits, blouses, skirts, slacks, etc.), hairstyles, jewelry, and soforth.

After identifying a collection of appropriate augmented realityelements, the augmented reality system 100 begins to narrow thecollection by applying any rules or preferences pre-defined by the user110 b. For example, in at least one embodiment, upon initially optinginto the services and features of the augmented reality system 100, theuser 110 b pre-defined one or more rules and/or preferences for anyfuture augmented reality overlays that would be displayed in combinationwith a view of her. To illustrate, the user 110 b may have configuredrules such as: “Always show me in women's clothing,” “Never show mewearing orange,” “Never change my hair color,” and so forth. Similarly,the user 110 b may have configured preferences such as: “I prefer towear darker colors,” “I prefer conservative hairstyles,” “I prefer towear mid-range fashion,” and so forth. Accordingly, in at least oneembodiment, the augmented reality system 100 applies any pre-configuredrules or preferences to the collection of appropriate augmented realityelements and calculates an initial score for each element in thecollection based on the applied rules or preferences.

Next, in one or more embodiments, the augmented reality system 100weights the initial score calculated for each augmented reality elementbased on networking system information. For example, the augmentedreality system 100 adjusts the initial scores based on social networkingactivity information associated with the user 110 b. To illustrate, ifthe user 110 b has previously “liked” a particular fashion designer viathe networking system 108, the augmented reality system 100 adds anamount to the initial score calculated for elements representing attireassociated with that fashion designer. Similarly, if the user 110 b aspreviously posted via the networking system 108 that she dislikes gaudyjewelry, the augmented reality system 100 subtracts an amount from theinitial score calculated for elements representing large and flashyjewelry.

As mentioned above, in at least one embodiment, the augmented realitysystem 100 further weights the score calculated for each identifiedaugmented reality element based on a networking system coefficientbetween the user 110 a and the user 110 b. For example, in one or moreembodiments, the networking system coefficient is a numericalrepresentation of the strength of the relationship between the users 110a and 110 b via the networking system 108. To illustrate, in oneembodiment the relationship coefficient between the users 110 a and 110b is high. If the networking system activity history associated with theuser 110 a indicates an affinity for a particular clothing style, theaugmented reality system 100 can add a certain amount to score foridentified augmented reality elements representing pieces of clothing inthat clothing style, even though the augmented reality system 100 iscalculating the score for those augmented reality elements relative tothe user 110 b.

After calculating a score for each of the collection of identifiedaugmented reality elements, the augmented reality system 100 beginsbuilding an augmented reality overlay for the user 110 b. For example,the augmented reality system 100 identifies the highest scoringaugmented reality elements to attach to the graphical mesh in order tobuild the augmented reality overlay. In one or more embodiments, theaugmented reality system 100 attaches the highest scoring augmentedreality elements to the graphical mesh based on a set of reality-basedrules. For instance, the augmented reality system 100 may operate underreality-based rules including rules that dictate how many articles ofclothing a person can wear (e.g., “a person can only wear one pair ofpants at a time,”), modesty rules (e.g., “a person's swimsuit areas mustbe covered”), and so forth. Thus, the augmented reality system 100 canattach the highest scoring augmented reality element to the graphicalmesh, then continue to the next highest scoring augmented realityelement based on the reality-based rules, and so on until a thresholdnumber of augmented reality elements have been attached. For example,the user 110 b may have previously specified a maximum number ofaugmented reality elements that can be included in an augmented realityoverlay, of the augmented reality system 100 may stop attachingaugmented reality elements to the graphical mesh when a thresholdpercentage of the graphical mesh has been covered.

In one or more embodiments, the augmented reality system 100 attaches anaugmented reality element to the graphical mesh by first determiningwhere on the graphical mesh the element should be attached. For example,in one embodiment, each augmented reality element is associated withmetadata that specifies a body area where the augmented reality elementshould be attached. Based on this information, the augmented realitysystem 100 attaches the augmented reality element to the appropriatearea within the graphical mesh. Next, the augmented reality system 100contours the attached element to the graphical mesh such that thegraphical mesh appears to be “wearing” the attached element. Forexample, the augmented reality system 100 can shorten trousers, lengthensleeves, widen waistbands, adjust hairlines, etc.

Once the augmented reality system 100 has attached the appropriatenumber of augmented reality elements to the graphical mesh, theaugmented reality overlay is built, and the augmented reality system 100can provide the augmented reality overlay to the augmented realitydisplay device 102 a. In response to receiving the augmented realityoverlay, the augmented reality display device 102 a anchors theaugmented reality overlay within a view of the user 110 b, such that theuser 110 b appears to be wearing the augmented reality overlay. In otherwords, the augmented reality display device 102 a displays the augmentedreality overlay such that portions of the user 110 b are completelyobscured by the augmented reality overlay.

To illustrate, as shown in FIG. 2B, the user 110 b is wearing jeans anda t-shirt, in addition to the augmented reality display device 102 b. Inresponse to performing the analysis and selection process describedabove, the augmented reality system 100 builds and provides theaugmented reality overlay 206 a, which the augmented reality displaydevice 102 a anchors within the view 204 of the user 110 b. As shown inFIG. 2B, the augmented reality display device 102 a anchors theaugmented reality overlay 206 a such that portions of the user 110 b arecompletely obscured.

For example, in response to analyzing the location, setting, andnetworking system information described above, the augmented realitysystem 100 has built the augmented reality overlay 206 a out of theaugmented reality elements 208 a, 208 b, 208 c, and 208 d. As shown inFIG. 2C, the augmented reality element 208 a represents anoffice-appropriate blouse, and the augmented reality element 208 brepresents a similarly appropriate skirt.

As mentioned above, in some embodiments, an augmented reality elementmay be an effect rather than a digital representation of an item (e.g.,a blouse, a skirt, etc.). For example, as shown in FIG. 2C, theaugmented reality element 208 c is a pair of uncovered legs. As shown inFIG. 2B, in real-life the user 110 b is wearing jeans (e.g., as shown inFIG. 2B). Thus, in order for her to appear to be wearing a skirt, theaugmented reality system 100 included the augmented reality element 208c. In one or more embodiments, the augmented reality system 100 performsskin tone matching when attaching this type of augmented reality elementto the graphical mesh associated with the user 110 b.

Similarly, the augmented reality element 208 d is a skin-tone overlaythat covers the augmented reality display device 102 b (as shown in FIG.2B). For example, the user 110 b may have pre-defined a preference thatstates she prefers to be shown without any eyewear. Accordingly, theaugmented reality system 100 performs skin tone matching when attachingthis type of augmented reality element to the graphical mesh associatedwith the user 110 b. Similarly, the augmented reality system 100 maydetermine the shape and color of her eyes by analyzing networking systeminformation (e.g., profile pictures, etc.). By utilizing the graphicalmesh representing the 3-dimensional shape of the user 110 b, theaugmented reality overlay 206 a appears within the view 204 to closelyapproximate the shape and contours of the user 110 b such that theresulting effect is not distracting to the user 110 a and the user 110 bappears to be wearing an entirely different outfit that is appropriateto the location and setting of both of the users 110 a, 110 b.

As discussed above, the augmented reality system 100 enables a user tomove from one event to the next without having to worry about changingclothes, re-doing makeup, shaving, fixing hair, and so forth. Forexample, in one embodiment, the users 110 a and 110 b traveled directlyfrom the board meeting (e.g., described with reference to FIGS. 2B and2C) out to have drinks at an upscale bar. Accordingly, in response toidentifying a new location and setting (e.g., an upscale bar in theevening), the augmented reality system 100 generates a new augmentedreality overlay for the user 110 b, in the manner described above, withaugmented reality elements that are appropriate to the new location andsetting.

For example, as shown in FIG. 2D, the augmented reality system 100provides the augmented reality overlay 206 b to the augmented realitydisplay device 102 a to be anchored on the view 204 of the user 110 b.As shown, the augmented reality overlay 206 b includes the augmentedreality elements 208 c, 208 e, 208 f, 208 g, and 208 h. In response todetermining that the augmented reality overlay 206 b would include theaugmented reality element 208 e (e.g., a sleeveless dress), theaugmented reality system 100 has also included the augmented realityelement 208 c (e.g., bare legs), the augmented reality element 208 h(e.g., bare shoulders), and the augmented reality element 208 i (e.g.,décolletage) in the augmented reality overlay 206 b.

Additionally, based on a combination of user preferences and othernetworking system information (e.g., networking system activityinformation indicating the user 110 b has “liked” a picture showing abobbed haircut), the augmented reality system 100 has also included theaugmented reality element 208 f As shown in FIG. 2D, the augmentedreality element 208 f is an effect that causes the hairstyle of the user110 b to change from long and wavy to bobbed and straight. Similarly,based on a combination of user preferences and other networking systeminformation (e.g., networking system activity information indication theuser 110 b has frequently “liked” pictures of a celebrity who alwayswears sunglasses in public), the augmented reality system 100 hasincluded the augmented reality element 208 g, which changes theappearance of the augmented reality display device 102 b to appear likesunglasses.

As described above, the augmented reality display device 102 a anchorsthe augmented reality overlay 206 b within the view 204 of the user 110b such that any movement within the view 204 is compensated for. Forexample, the augmented reality display device 102 a can detect movementsof the user 110 a and/or the user 110 b and can reposition the augmentedreality overlay 206 b in a corresponding manner such that portions ofthe user 110 b are effectively obscured. Thus, from the perspective ofthe user 110 a, the user 110 b appears to be wearing a cocktail dress ina look that is appropriate for this new location and setting.

In one or more embodiments, the augmented reality system 100 generatesdifferent augmented reality overlays for the same person depending onthe other augmented reality system users who are looking at that person.For example, as illustrated in FIG. 3A, the users 110 a, 110 b, and 110c may be in the same location and setting. Even though the users 110 a,110 b, and 110 c are similarly located, the networking systemrelationship coefficient between the user 110 a and the user 110 b maybe different from the networking system relationship coefficient betweenthe user 110 c and the user 110 b. For instance, the user 110 a may bethe employer of the user 110 b, while the user 110 c is a long-timefriend of the user 110 b. Accordingly, in that embodiment, thenetworking system relationship coefficient between the user 110 b andthe user 110 c is higher than the networking system relationshipcoefficient between the user 110 b and the user 110 a. Thus, when boththe users 110 a and 110 c are looking at the user 110 b through theaugmented reality display devices 102 a and 102 c, respectively, theaugmented reality system 100 can generate two different augmentedreality overlays associated with the user 110 b.

For example, after performing the analysis of the location, setting, andnetworking system information described above in combination with therelationship coefficient between the user 110 a and the user 110 b, theaugmented reality system 100 can generate and provide the augmentedreality overlay 206 c, illustrated in FIG. 3B. In FIG. 3B, the augmentedreality system 100 anchors the augmented reality overlay 206 c in theview 302 provided by the augmented reality display device 102 aassociated with the user 110 a. As shown in FIG. 3B, the augmentedreality overlay 206 c includes the augmented reality elements 208 c, 208d, 208 h, 208 i, and 208 j. As described above with reference to FIGS.2B and 2C, in response to determining that the augmented reality element208 j (e.g., a sleeveless dress) has a calculated score that is highenough to warrant inclusion in the augmented reality overlay 206 c, theaugmented reality system 100 also includes the augmented reality element208 c (e.g., bare legs), the augmented reality element 208 h (e.g., bareshoulders), and the augmented reality element 208 i (e.g., décolletage)in the augmented reality overlay 206 b. Similarly, in response to one ormore preferences pre-defined by the user 110 b, the augmented realitysystem 100 also includes the augmented reality element 208 d (e.g., anoverlay that masks the augmented reality display device 102 b worn bythe user 110 b).

In response to determining that the networking system relationshipcoefficient between the user 110 b and the user 110 c is higher, theaugmented reality system 100 can generate a different augmented realityoverlay for display to the user 110 c. Alternatively, in at least oneembodiment, the augmented reality system 100 simply adds additionalaugmented reality elements to a previously generated augmented realityoverlay that reflect the closer relationship indicated by the higherrelationship coefficient. For example, as illustrated in FIG. 3C, theaugmented reality system 100 can add the augmented reality element 208 kto the augmented reality overlay 206 c anchored within a view 304 of theuser 110 b provided to the user 110 c by the augmented reality displaydevice 102 c.

In one or more embodiments, the augmented reality system 100 adds theaugmented reality element 208 k to the augmented reality overlay 206 cin response to determining that the user 110 b and the user 110 c have anetworking system relationship coefficient that is higher than athreshold amount. For example, in at least one embodiment, arelationship coefficient between two users that is higher than thethreshold amount indicates to the augmented reality system 100 that thetwo users have a strong relationship. To illustrate, in one embodiment,the users 110 b and 110 c have a high relationship coefficient that theaugmented reality system 100 has calculated based on the length of their“friendship” via the networking system 108, their frequent shared posts,their frequent shared check-ins, their frequent cross-tags in digitalphotographs and videos, and so forth. Additionally, the augmentedreality system 100 has calculated the high relationship coefficient forthe users 110 b, 110 c based on their profile information that indicatesthey live in the same area and share an interest in the same sportsteam. Accordingly, in light of this analysis, the augmented realitysystem 100 has added the augmented reality element 208 k (e.g., a hatfeaturing the logo of the sports team in which both of the users 110 b,110 c are interested) to the augmented reality overlay 206 c.

Thus, in one or more embodiments, the augmented reality system 100provides a unique view to each augmented reality system user. Forexample, even though the view 302 (e.g., as shown in FIG. 3B) and theview 304 (e.g., as shown in FIG. 3C) are of the same user (e.g., theuser 110 b) at the same time and in the same place, the augmentedreality system 100 provides a different combination of augmented realityelements in the displayed overlay. In at least one embodiment, if thereis no relationship coefficient between a group of augmented realityusers (e.g., no one in the group shares any networking system activityor information), the augmented reality system 100 may provide the sameaugmented reality elements in an augmented reality overlay for aparticular member of the group. In that embodiment, that particularmember would appear the same to every other group member who looks ather.

As mentioned above, the augmented reality system 100 also provides voicetranslation services. For example, the augmented reality system 100includes a machine translator that analyzes an audio stream for spokenwords, determines the language associated with the words, translates thewords into a different language, and provides an audio stream of thespoken translated words. In at least one embodiment, the augmentedreality system 100 determines the language associated with an incomingaudio stream and the appropriate translated language based, at least inpart, on networking system information.

To illustrate, the user 110 a may speak French while the user 110 bspeaks English. In one or more embodiments, the augmented reality system100 determines the languages of the user 110 a and the user 110 b basedon their networking system profile information, their networking systemcheck-ins, their networking system posts, their networking systemcomments, and so forth. Accordingly, when the augmented reality displaydevice 102 a provides an image including the user 110 b to the augmentedreality system 100, the augmented reality system 100 can performtranslation services utilizes the audio stream provided by the augmentedreality display device 102 a. The augmented reality system 100 thenprovides an audio stream of translated speech back to the augmentedreality display device 102 a for playback to the user 110 a.

Further, in one or more embodiments, when providing an audio stream oftranslated speech, the augmented reality system 100 can also include anaugmented reality element in an augmented reality overlay that isassociated with the translation. For example, in one embodiment, inresponse to providing an audio stream of translated speech to theaugmented reality display device 102 a, the augmented reality system 100provides an augmented reality element that makes it appear like themouth of the user 110 b is synchronized to the translated speech.

As discussed above, the systems and methods laid out with reference toFIGS. 1-3 provide a augmented reality overlay for a networking systemuser, such that the user appears to others to be in the correct attireand speaking the correct language for a given situation. FIG. 4illustrates a schematic diagram illustrating another example embodimentof the augmented reality system 100. As shown in FIG. 4, the augmentedreality system 100 includes, but is not limited to, the augmentedreality display devices 102 a, 102 b, the server device(s) 106, and theaugmented reality marketplace 114. In one or more embodiments, theaugmented reality display devices 102 a, 102 b include networking systemapplications 104 a, 104 b, respectively. As shown in FIG. 4, thenetworking system application 104 a, 104 b includes a communicationmanager 402 a, 402 b, a display manager 404 a, 404 b, an audio manager406 a, 406 b, and a device manager 408 a, 408 b. Furthermore, in one ormore embodiments, the augmented reality display devices 102 a, 102 binclude an audio/visual transmission device, an augmented realitydisplay, a video capturing device, and an audio playback device.

Additionally shown in FIG. 4, the server device(s) 106 hosts thenetworking system 108. In one or more embodiments, the networking system108 includes a networking system activity manager 410, an augmentedreality overlay engine 412, a translation manager 414, and a datastorage 416. As shown in FIG. 4, the data storage 416 includes augmentedreality data 418.

In at least one embodiment, the augmented reality system 100 accessesthe networking system 108 in order to identify and analyze networkingsystem user data. Accordingly, the networking system 108 includes asocial graph 420 for representing a plurality of users, actions, andconcepts. In one or more embodiments, the social graph 420 includes nodeinformation 422 and edge information 424. Node information 422 of thesocial graph 420 stores information including, for example, nodes forusers and nodes for repositories. Edge information 424 of the socialgraph 420 stores information including relationships between nodesand/or actions occurring within the networking system 108. Furtherdetails regarding the networking system 108, the social graph 420,edges, and nodes are presented below with respect to FIG. 8.

Each of the components 410-420 of the networking system 108 can beimplemented using a computing device including at least one processorexecuting instructions that cause the augmented reality system 100 toperform the processes described herein. In some embodiments, thenetworking system components described herein can be implemented by theserver device(s) 106, or across multiple server devices. Additionally oralternatively a combination of one or more server devices and one ormore client-computing devices can implemented components of thenetworking system 108. Additionally or alternatively, the componentsdescribed herein can comprise a combination of computer-executableinstructions and hardware.

As mentioned above, and as shown in FIG. 4, the networking systemapplication 104 a, 104 b includes the communication manager 402 a, 402b. In one or more embodiments, the communication manager 402 a, 402 bsends and receives data to and from the networking system 108. Forexample, the communication manager 402 a, 402 b sends data to thenetworking system 108 including a video stream or an image captured bythe video capturing device of the augmented reality display device 102a, 102 b. Additionally, the communication manager 402 a, 402 b sendsother types of information, such as GPS data associated with theaugmented reality display device 102 a, 102 b. Further, thecommunication manager 402 a, 402 b receives data from the networkingsystem 108 including a augmented reality overlay associated with anotheraugmented reality system user at whom the user of the augmented realitydisplay device 102 a, 102 b is looking. In at least one embodiment, thecommunication manager 402 a, 402 b also receives data from thenetworking system 108 including an audio stream of translated speechand/or other sounds (e.g., ambient sounds, music, etc.).

As shown in FIG. 4, and as mentioned above, the networking systemapplication 104 a, 104 b includes the display manager 404 a, 404 b. Inone or more embodiments, the display manager 404 a, 404 b anchors areceived augmented reality overlay within the augmented reality display.For example, in order to effectively mask some or all of a user'sappearance as viewed through the augmented reality display, the displaymanager 404 a, 404 b can determine where to anchor the receivedaugmented reality overlay by utilizing pattern or shape matching (e.g.,matching the shape of the augmented reality overlay to the shape of theuser).

After anchoring the augmented reality overlay, the display manager 404a, 404 b also displays the augmented reality overlay. For example, inone or more embodiments, the display manager 404 a, 404 b displays theaugmented reality overlay within the augmented reality display device102 a, 102 b by projecting the augmented reality overlay onto a lens orscreen associated with the augmented reality display device 102 a, 102b. Regardless of the display method utilized by the display manager 404a, 404 b, the display manager 404 a, 404 b displays the augmentedreality overlay such that a view of the surrounding environment throughthe display of the augmented reality display device 102 a, 102 b is notobstructed.

Once the display manager 404 a, 404 b determines where to anchor theaugmented reality overlay, the display manager 404 a, 404 b continues todisplay the augmented reality overlay while compensating for movement inthe display. For example, the user of the augmented reality displaydevice 102 a may move his head slightly while looking at the user of theaugmented reality display device 102 b. Accordingly, the display manager404 a senses the movement (e.g., by image analysis, by gyroscope, etc.),and repositions the augmented reality overlay within the augmentedreality display of the augmented reality display device 102 a. In thisway, the display manager 404 a, 404 b ensures correct positioning of aaugmented reality overlay within the augmented reality displayregardless of any movement of the augmented reality display device 102a, 102 b.

As shown in FIG. 4, and as mentioned above, the networking systemapplication 104 a, 104 b includes the audio manager 406 a, 406 b. In oneor more embodiments, the audio manager 406 a, 406 b provides audiothrough an audio playback device associated with the augmented realitydisplay device 102 a, 102 b (e.g., headphones, etc.) that is receivedfrom the networking system 108. As mentioned above, in some embodiments,the augmented reality system 100 provides real-time languagetranslation. For example, a first user may speak to a second user inRussian. The augmented reality system 100 can utilize networking systeminformation associated with the second user to determine that the seconduser only speaks English, and can provide an English translation to thesecond user of what the first user is saying in Russian. Accordingly, inone or more embodiments, the audio manager 406 a, 406 b plays a receivedaudio stream including machine translated synthetic speech to the userof the augmented reality display device 102 a, 102 b via an audioplayback device associated with the augmented reality display device 102a, 102 b.

In some embodiments, the audio manager 406 a, 406 b also performs noisecancelation in connection with an audio playback device associated withthe augmented reality display device 102 a, 102 b. In order to prevent afirst user from hearing both machine translated speech and the voice ofthe second user (e.g., the user whose speech is being translated), theaudio manager 406 a, 406 b can provide noise cancelation in addition tothe playback of the audio stream provided by the networking system 108.In one or more embodiments, the noise cancelation effectively mutes anybackground sounds that may interference with the sound quality of theincoming audio stream.

As shown in FIG. 4, and as mentioned above, the networking systemapplication 104 a, 104 b includes a device manager 408 a, 408 b. In oneor more embodiments, the device manager 408 a, 408 b handles auxiliaryfeatures of the augmented reality display device 102 a, 102 b that arenecessary to the operation of the augmented reality system 100. Forexample, in one embodiment, the augmented reality system 100 utilizesglobal positioning satellite (“GPS”) data associated with the augmentedreality display device 102 a, 102 b to determine where an augmentedreality system 100 user is currently located. Accordingly, the devicemanager 408 a, 408 b queries location data from a GPS unit associatedwith the augmented reality display device 102 a, 102 b. In additional oralternative embodiments, the device manager 408 a, 408 b queries andcommunicates data including, but not limited to, WiFi data, batteryusage data, and so forth.

Also shown in FIG. 2, and mentioned above, the server device(s) 106hosts the networking system 108. The networking system 108 generates andprovides augmented reality overlays to the augmented reality displaydevices 102 a, 102 b. Additionally, the networking system 108 providesconventional networking system features such as, but not limited to,newsfeeds, posts, electronic messages, and so forth. For example, thenetworking system 108 can provide a networking system user with anewsfeed including posts submitted by the user's networking system“friends.”

Further shown in FIG. 2, the networking system 108 includes thenetworking system activity manager 410. As mentioned above, theaugmented reality system 100 automatically generates an augmentedreality overlay associated with a user based, in part, on the user'snetworking system activities. For example, the user may engage innetworking activities such as, but not limited to, submitting posts,sharing posts, liking posts, submitting and watching live video,commenting on posts and digital media, messaging other users, checkingin at locations, tagging other users in posts, submitting payments, etc.In one or more embodiments, the networking system activity manager 410monitors and tracks all networking system activities engaged in by anetworking system user.

The networking system activity manager 410 also calculates a networkingsystem relationship coefficient between two users. As mentioned above,the augmented reality system 100 can generate different augmentedreality overlays for a user depending on who is looking at him. Forexample, the augmented reality system 100 provides a different augmentedreality overlay to the user's employer than the augmented realityoverlay provided to the user's friend, even though the user, his friend,and his employer are all at the same location. In one or moreembodiments, the networking system activity manager 410 calculates arelationship coefficient between two users based on the networkingsystem activity the two users have in common. For example, thenetworking system activity manager 410 may calculate a high relationshipcoefficient for two users who frequently tag each other in posts,check-in at the same locations at the same times, share each other'sposts, and so forth.

The networking system activity manager 410 can also take additionalinformation into account when calculating a relationship coefficient.For example, the networking system activity manager 410 can analyzeprofile and account information for both users. In one or moreembodiments, the users' profile and account information may includerelationship information (e.g., the users are related), the length oftheir “friendship” (e.g., they have been friends via the networkingsystem 108 for a given amount of time), how many networking systemfriends the users have in common, etc.

As shown in FIG. 4, and as mentioned above, the networking system 108includes the augmented reality overlay engine 412. In one or moreembodiments, the augmented reality overlay engine 412 handles all tasksrelated to building and providing an augmented reality overlay to anetworking system user. For example, in at least one embodiment, theaugmented reality overlay engine 412 performs facial recognition on animage from a received video stream in order to identify a networkingsystem user. As discussed above, the networking system 108 receives avideo stream or single image from the augmented reality display device102 a (e.g., including whatever the user of the augmented realitydisplay device 102 a is looking at). Accordingly, the augmented realityoverlay engine 412 extracts an image frame from the video stream andutilizes various computer vision techniques to determine whether theimage frame includes a person. If the image frame includes a person, theaugmented reality overlay engine 412 performs facial recognition todetermine whether the person in the image frame is a networking systemuser. If the augmented reality overlay engine 412 is unable to match theperson in the image frame to any image of a networking system user, theaugmented reality overlay engine 412 does not continue in the process ofgenerating a augmented reality overlay. Alternatively, the augmentedreality overlay engine 412 can provide a default augmented realityoverlay.

After determining the networking system identity of the person at whomthe user of the augmented reality display device 102 a is looking (e.g.,the person in the image frame from the received video stream), theaugmented reality overlay engine 412 performs a 3D scan of the imageframe in order to determine the 3-dimensional shape of the person in theimage frame. For example, the augmented reality overlay engine 412creates a point cloud of data from the displayed person's 2-dimensionalsurface within the image frame. From this point cloud, the augmentedreality overlay engine 412 then builds a graphical mesh that representsthe displayed person's 3-dimensional shape. This graphical mesh becomesthe scaffold to which the augmented reality overlay engine 412 canattach augmented reality elements in order to generate an augmentedreality overlay.

Once the graphical mesh for the displayed person is built, the augmentedreality overlay engine 412 identifies augmented reality elements toinclude in a augmented reality overlay for the displayed person (e.g.,the user of the augmented reality display device 102 b). In one or moreembodiments, the process for identifying augmented reality elementsbegins with identifying augmented reality elements that are appropriateto the location of the augmented reality display devices 102 a, 102 b.As discussed above, the augmented reality display devices 102 a, 102 bcan provide GPS information, WiFi information, and other locationinformation to the networking system 108. The augmented reality overlayengine 412 utilizes this location information to determine the locationof the augmented reality display devices 102 a, 102 b.

In one or more embodiments, the augmented reality overlay engine 412utilizes networking system information, Internet lookups, databasesearches, etc. to identify appropriate augmented reality elements basedon the location of the augmented reality display devices 102 a, 102 b.For example, the augmented reality overlay engine 412 can utilize theGPS location of the augmented reality display devices 102 a, 102 b incombination with networking system information to determine theaugmented reality display devices 102 a, 102 b are located at a birthdayparty at an upscale restaurant. In at least one embodiment, theaugmented reality overlay engine 412 can analyze web search results(e.g., press articles, digital photographs, patron reviews, etc.) todetermine what kinds of attire people generally wear to the upscalerestaurant. The augmented reality overlay engine 412 can then selectaugmented reality elements from a library or collection of augmentedreality elements that are similar to those kinds of attire. Forinstance, the augmented reality overlay engine 412 can selectappropriate augmented reality elements by performing image comparisons,analyzing metadata associated with each augmented reality element, andso forth.

At this point in the augmented reality overlay building process, theaugmented reality overlay engine 412 may have identified hundreds ofaugmented reality elements that are appropriate for the location of theaugmented reality display devices 102 a, 102 b. Accordingly, in one ormore embodiments, the augmented reality overlay engine 412 narrows thecollection of identified augmented reality elements by calculating ascore for each identified augmented reality element that is based onuser preferences, networking system information, and a networking systemrelationship coefficient between the users of the augmented realitydisplay devices 102 a, 102 b. The augmented reality overlay engine 412scores the identified augmented reality elements such that the highestscoring augmented reality elements are most likely to be those thedisplayed person (e.g., the user of the augmented reality display device102 b) would have chosen himself.

In at least one embodiment, the augmented reality overlay engine 412begins narrowing the collection of identified augmented reality elementsby applying any rules or preferences that have been pre-configured bythe displayed person (e.g., the user 110 b of the augmented realitydisplay device 102 b). As mentioned above, an augmented reality systemuser can pre-configure rules or preferences that the augmented realitysystem 100 applies when generating a augmented reality overlay for thatuser. For example, the user can configure rules or preferences such as:“always show me in tops or dresses with sleeves,” “never show me withfacial hair,” “I don't like heavy makeup,” “I typically don't wear greysuits,” etc.

Accordingly, in one or more embodiments, the augmented reality overlayengine 412 applies the pre-configured rules or preferences of thedisplayed person to the identified augmented reality elements in orderto determine a score for each of the augmented reality elements. Forexample, the augmented reality overlay engine 412 can calculate a higherscore for those augmented reality elements that are in keeping with thedisplayed person's rules or preferences. Similarly, the augmentedreality overlay engine 412 can calculate a lower score for thoseaugmented reality elements that are not in keeping with the displayedperson's rules or preferences. In at least one embodiment, the augmentedreality overlay engine 412 removes those elements from the collection ofidentified augmented reality elements that fail to have a calculatedscore above a threshold amount. In at least one embodiment, theaugmented reality overlay engine 412 calculates a score for an augmentedreality element based on rules or preferences by utilizing computervision in connection with the augmented reality elements, by analyzingmetadata associated with the augmented reality elements, by usingmachine learning techniques, and so forth.

Next, the augmented reality overlay engine 412 weights the calculatedscores for the identified augmented reality elements based on therelationship coefficient between the displayed person (e.g., the user ofthe augmented reality display device 102 b) and the user looking at thedisplayed person (e.g., the user of the augmented reality display device102 a). As discussed above, the networking system activity manager 410calculates the networking system relationship coefficient between twonetworking system users based on a variety of factors including, but notlimited to, the users' networking system activity history, the users'networking system interactions, the users' networking system profileinformation, etc. When the relationship coefficient between the users ofthe augmented reality display devices 102 a, 102 b is high, theaugmented reality overlay engine 412 adds a heavier weight to the scoresof augmented reality elements that correlate with networking systeminformation upon which the relationship coefficient was calculated.

In one example, the users of the augmented reality display devices 102a, 102 b have a higher relationship coefficient based on networkingsystem information including frequent networking system posts on eachother's newsfeeds regarding the same sports team, several check-ins atevents for the same sports team, profile information indicating anaffinity for the same sports team. Accordingly, when the users of theaugmented reality display devices 102 a, 102 b are at a sports bar onthe same night as a game where that particular sports team is playing,the augmented reality overlay engine 412 will add a heavy weight to thecalculated scores for augmented reality elements that are associatedwith that team (e.g., elements that include the team's logo, elementsthat are the same colors as the team's colors, etc.).

In another example, the users of the augmented reality display devices102 a, 102 b have a lower relationship coefficient based on networkingsystem information that indicates the users infrequently interact viathe networking system 108, but are employed at the same company.Accordingly, when the users of the augmented reality display devices 102a, 102 b are at the sports bar on the same night, the augmented realityoverlay engine 412 may only add weight to calculated scores foraugmented reality elements that correlate with common interests of usersat that location, users who are also employed at the same company as theusers of the augmented reality display devices 102 a, 102 b, and soforth. In this way, the augmented reality system 100 may generate acompletely different augmented reality overlay for a user, depending onwho is looking at him.

Once the augmented reality overlay engine 412 has calculated weightedscores for each of the identified augmented reality elements, theaugmented reality overlay engine 412 can begin building an augmentedreality overlay for the displayed person (e.g., the user of theaugmented reality display device 102 b) with the highest scoringaugmented reality elements. For example, in one embodiment, theaugmented reality overlay engine 412 selects all augmented realityelements with a weighted score above a threshold amount to include inthe augmented reality overlay. In another embodiment, the augmentedreality overlay engine 412 selects a top number (e.g., the top ten) ofthe highest scoring augmented reality elements to include in theaugmented reality overlay.

In some embodiments, the augmented reality overlay engine 412 operatesunder a set of realistic constraints when selecting augmented realityelements to include in the augmented reality overlay. For example, inone embodiment, the highest scoring augmented reality elements mayinclude two dresses. This is problematic because in real-life a personwould not wear two dresses at the same time. In order to keep theaugmented reality overlay engine 412 from selecting augmented realityelements to include in the augmented reality overlay that areincompatible with each other, the augmented reality overlay engine 412operates under constraints that are informed by real-world scenarios(e.g., “a person only wears one article of clothing over each area ofthe body,” “a person does not carry an open umbrella indoors,” “a persondoes not wear snow boots in the Summer,” etc.). In at least oneembodiment, the augmented reality overlay engine 412 is furtherconstrained by general user preferences (e.g., “I want my augmentedreality overlays to include no more than five elements”).

With an appropriate number of compatible augmented reality elementsselected, the augmented reality overlay engine 412 begins building theaugmented reality overlay by attaching the augmented reality elements tothe display mesh generated for the displayed user (e.g., the user of theaugmented reality display device 102 b). In one or more embodiments, inorder to attach an augmented reality element to the display mesh, theaugmented reality overlay engine 412 may resize the augmented realityelement, may contour the augmented reality element, may add shadow to aportion of the augmented reality element, and so forth in order to makeit appear as though the augmented reality element is being worn.Accordingly, the augmented reality overlay engine 412 continuesattaching the augmented reality elements to the display mesh until theaugmented reality overlay is completely built. In some embodiments, theaugmented reality overlay may only include as little as one augmentedreality element. In other embodiments, the augmented reality overlay mayinclude so many augmented reality elements that when overlaid on thedisplayed person, the augmented reality overlay completely coversportions of the displayed person.

At this point, the augmented reality overlay engine 412 provides theaugmented reality overlay to the augmented reality display device 102 ato be overlaid on the displayed person within the augmented realitydisplay of the augmented reality display device 102 a. As describedabove, the display manager 404 a of the augmented reality display device102 a anchors the augmented reality overlay within the augmented realitydisplay such that the augmented reality overlay moves when the user ofthe augmented reality display device 102 b moves. In this way, from theperspective of the user of the augmented reality display device 102 a,the user of the augmented reality display device 102 b appears to beattired in whatever augmented reality elements are included in theaugmented reality overlay.

As mentioned above, and as illustrated in FIG. 4, the networking system108 further includes the translation manager 414. In one or moreembodiments, the augmented reality system 100 provides translationservices that enable the users of the augmented reality display devices102 a, 102 b to communicate even if they do not share a language.Accordingly, in at least one embodiment, the translation manager 414receives an audio stream from the augmented reality display device 102 bincluding the user of the augmented reality display device 102 bspeaking, and provides an audio stream of translated speech to theaugmented reality display device 102 a (and vice versa). The translationmanager 414 may provide translated audio into various languages based onuser settings, or may automatically determine a user's languagepreferences based on networking system information associated with theuser.

Furthermore, in order to aid in the communication between the users ofthe augmented reality display devices 102 a, 102 b, the translationmanager 414 can generate and add an augmented reality element to theaugmented reality overlay generated by the augmented reality overlayengine 412. For example, in one or more embodiments, the translationmanager 414 can generate an augmented reality element that looks likethe displayed user's mouth. Furthermore, the translation manager 414 cansynchronize the generated mouth with the audio stream of translatedspeech. Thus, when attached to the augmented reality overlay, thisaugmented reality element makes it appear as though the displayed useris speaking the words included in the audio stream of translated speech.In one or more embodiments, the translation manager 414 generates thisaugmented reality element by matching the displayed person's skin toneand facial features (e.g., facial hair, mouth shape, birth marks, etc.).

As further shown in FIG. 4, and as mentioned above, the networkingsystem 108 includes the data storage 416. The data storage 416 includesaugmented reality data 418. In one or more embodiments, the augmentedreality data 418 is representative of augmented reality information(e.g., the library of augmented reality elements, generated augmentedreality overlays, user rules and preferences, networking systeminformation), such as described herein.

FIGS. 1-4, the corresponding text and examples, provide a number ofdifferent methods, systems, and devices for providing a augmentedreality overlay within a view provided by an augmented reality displaydevice. In addition to the foregoing, embodiments can also be describedin terms of flowcharts comprising acts and steps in a method foraccomplishing a particular result. For example, FIG. 5 may be performedwith less or more steps/acts or the steps/acts may be performed indiffering orders. Additionally, the steps/acts described herein may berepeated or performed in parallel with one another or in parallel withdifferent instances of the same or similar steps/acts.

FIG. 5 illustrates a flowchart of one example method 500 of providing anaugmented reality overlay. The method 500 includes an act 510 ofreceiving one or more images. In particular, the act 510 can involvereceiving one or more images captured by an augmented reality displaydevice associated with a networking system user. For example, the one ormore images may be captured by a recording device associated with theaugmented reality display device.

The method 500 also includes an act 520 of determining an identity of anetworking system co-user. In particular, the act 520 can involvedetermining an identity of a networking system co-user shown in the oneor more images. For example, determining an identity of the networkingsystem co-user can include utilizing, in association with the one ormore images, one or more of facial recognition techniques, imagematching, or eye scanning.

Additionally, the method 500 can include acts of calculating thenetworking system relationship coefficient associated with therelationship between the networking system user and the networkingsystem co-user, and generating an augmented reality overlay based, atleast in part, on the calculated networking system relationshipcoefficient. For example, calculating the networking system relationshipcoefficient can include calculating the networking system relationshipcoefficient based on networking system activity information commonbetween the networking system user and the networking system co-user.

Furthermore, the method 500 includes an act 530 of generating anaugmented reality overlay for the identified networking system co-user.In particular, the act 530 can involve generating, based on arelationship between the networking system user and the networkingsystem co-user, an augmented reality overlay for the identifiednetworking system co-user, wherein the augmented reality overlayincludes one or more augmented reality elements. In one embodiment,generating the augmented reality overlay for the identified networkingsystem co-user is further based on a location and setting associatedwith the augmented reality display device. For example, the method 500can include determining the location and setting associated with theaugmented reality display device based on one or more of GPSinformation, WiFi information, networking system information, and imageanalysis information from the one or more images.

In one or more embodiments, the method 500 further includes the act ofbuilding a graphical mesh based on a three-dimensional scan of thenetworking system co-user shown in the one or more images. For example,generating the augmented reality overlay for the identified networkingsystem co-user includes identifying one or more augmented realityelements that are appropriate to the location and setting, scoring theidentified one or more augmented reality elements based on predefinedrules associated with the networking system co-user, and weighting thescores for each of the identified one or more augmented reality elementsbased on the networking system relationship coefficient, and on thenetworking system information associated with the networking systemco-user. Additionally, generating the augmented reality overlay for theidentified networking system co-user can further include identifying aplurality of the scored augmented reality elements with weighted scoresabove a threshold amount, and attaching the plurality of augmentedreality element to the graphical mesh. For instance, attaching theplurality of augmented reality elements to the graphical mesh caninclude attaching the plurality of augmented reality elements based on aset of reality-based rules that ensure the networking system co-user'sappearance is realistic after the augmented reality overlay is displayedon the augmented reality display device.

Embodiments of the present disclosure may comprise or utilize a specialpurpose or general-purpose computer including computer hardware, suchas, for example, one or more processors and system memory, as discussedin greater detail below. Embodiments within the scope of the presentdisclosure also include physical and other computer-readable media forcarrying or storing computer-executable instructions and/or datastructures. In particular, one or more of the processes described hereinmay be implemented at least in part as instructions embodied in anon-transitory computer-readable medium and executable by one or morecomputing devices (e.g., any of the media content access devicesdescribed herein). In general, a processor (e.g., a microprocessor)receives instructions, from a non-transitory computer-readable medium,(e.g., a memory, etc.), and executes those instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein.

Computer-readable media can be any available media that can be accessedby a general purpose or special purpose computer system.Computer-readable media that store computer-executable instructions arenon-transitory computer-readable storage media (devices).Computer-readable media that carry computer-executable instructions aretransmission media. Thus, by way of example, and not limitation,embodiments of the disclosure can comprise at least two distinctlydifferent kinds of computer-readable media: non-transitorycomputer-readable storage media (devices) and transmission media.

Non-transitory computer-readable storage media (devices) includes RAM,ROM, EEPROM, CD-ROM, solid state drives (“SSDs”) (e.g., based on RAM),Flash memory, phase-change memory (“PCM”), other types of memory, otheroptical disk storage, magnetic disk storage or other magnetic storagedevices, or any other medium which can be used to store desired programcode means in the form of computer-executable instructions or datastructures and which can be accessed by a general purpose or specialpurpose computer.

A “network” is defined as one or more data links that enable thetransport of electronic data between computer systems and/or modulesand/or other electronic devices. When information is transferred orprovided over a network or another communications connection (eitherhardwired, wireless, or a combination of hardwired or wireless) to acomputer, the computer properly views the connection as a transmissionmedium. Transmissions media can include a network and/or data linkswhich can be used to carry desired program code means in the form ofcomputer-executable instructions or data structures and which can beaccessed by a general purpose or special purpose computer. Combinationsof the above should also be included within the scope ofcomputer-readable media.

Further, upon reaching various computer system components, program codemeans in the form of computer-executable instructions or data structurescan be transferred automatically from transmission media tonon-transitory computer-readable storage media (devices) (or viceversa). For example, computer-executable instructions or data structuresreceived over a network or data link can be buffered in RAM within anetwork interface module (e.g., a “NIC”), and then eventuallytransferred to computer system RAM and/or to less volatile computerstorage media (devices) at a computer system. Thus, it should beunderstood that non-transitory computer-readable storage media (devices)can be included in computer system components that also (or evenprimarily) utilize transmission media.

Computer-executable instructions comprise, for example, instructions anddata which, when executed at a processor, cause a general purposecomputer, special purpose computer, or special purpose processing deviceto perform a certain function or group of functions. In someembodiments, computer-executable instructions are executed on ageneral-purpose computer to turn the general-purpose computer into aspecial purpose computer implementing elements of the disclosure. Thecomputer executable instructions may be, for example, binaries,intermediate format instructions such as assembly language, or evensource code. Although the subject matter has been described in languagespecific to structural features and/or methodological acts, it is to beunderstood that the subject matter defined in the appended claims is notnecessarily limited to the described features or acts described above.Rather, the described features and acts are disclosed as example formsof implementing the claims.

Those skilled in the art will appreciate that the disclosure may bepracticed in network computing environments with many types of computersystem configurations, including, personal computers, desktop computers,laptop computers, message processors, hand-held devices, multi-processorsystems, microprocessor-based or programmable consumer electronics,network PCs, minicomputers, mainframe computers, mobile telephones,PDAs, tablets, pagers, routers, switches, and the like. The disclosuremay also be practiced in distributed system environments where local andremote computer systems, which are linked (either by hardwired datalinks, wireless data links, or by a combination of hardwired andwireless data links) through a network, both perform tasks. In adistributed system environment, program modules may be located in bothlocal and remote memory storage devices.

Embodiments of the present disclosure can also be implemented in cloudcomputing environments. In this description, “cloud computing” isdefined as a model for enabling on-demand network access to a sharedpool of configurable computing resources. For example, cloud computingcan be employed in the marketplace to offer ubiquitous and convenienton-demand access to the shared pool of configurable computing resources.The shared pool of configurable computing resources can be rapidlyprovisioned via virtualization and released with low management effortor service provider interaction, and then scaled accordingly.

A cloud-computing model can be composed of various characteristics suchas, for example, on-demand self-service, broad network access, resourcepooling, rapid elasticity, measured service, and so forth. Acloud-computing model can also expose various service models, such as,for example, Software as a Service (“SaaS”), Platform as a Service(“PaaS”), and Infrastructure as a Service (“IaaS”). A cloud-computingmodel can also be deployed using different deployment models such asprivate cloud, community cloud, public cloud, hybrid cloud, and soforth. In this description and in the claims, a “cloud-computingenvironment” is an environment in which cloud computing is employed.

FIG. 6 illustrates a block diagram of exemplary computing device 600that may be configured to perform one or more of the processes describedabove. One will appreciate that one or more computing devices such asthe computing device 600 may implement the augmented reality system 100.As shown by FIG. 6, the computing device 600 can comprise a processor602, a memory 604, a storage device 606, an I/O interface 608, and acommunication interface 610, which may be communicatively coupled by wayof a communication infrastructure 612. While an exemplary computingdevice 600 is shown in FIG. 6, the components illustrated in FIG. 6 arenot intended to be limiting. Additional or alternative components may beused in other embodiments. Furthermore, in certain embodiments, thecomputing device 600 can include fewer components than those shown inFIG. 6. Components of the computing device 600 shown in FIG. 6 will nowbe described in additional detail.

In one or more embodiments, the processor 602 includes hardware forexecuting instructions, such as those making up a computer program. Asan example and not by way of limitation, to execute instructions, theprocessor 602 may retrieve (or fetch) the instructions from an internalregister, an internal cache, the memory 604, or the storage device 606and decode and execute them. In one or more embodiments, the processor602 may include one or more internal caches for data, instructions, oraddresses. As an example and not by way of limitation, the processor 602may include one or more instruction caches, one or more data caches, andone or more translation lookaside buffers (TLBs). Instructions in theinstruction caches may be copies of instructions in the memory 604 orthe storage device 606.

The memory 604 may be used for storing data, metadata, and programs forexecution by the processor(s). The memory 604 may include one or more ofvolatile and non-volatile memories, such as Random Access Memory(“RAM”), Read Only Memory (“ROM”), a solid state disk (“SSD”), Flash,Phase Change Memory (“PCM”), or other types of data storage. The memory604 may be internal or distributed memory.

The storage device 606 includes storage for storing data orinstructions. As an example and not by way of limitation, storage device606 can comprise a non-transitory storage medium described above. Thestorage device 606 may include a hard disk drive (HDD), a floppy diskdrive, flash memory, an optical disc, a magneto-optical disc, magnetictape, or a Universal Serial Bus (USB) drive or a combination of two ormore of these. The storage device 606 may include removable ornon-removable (or fixed) media, where appropriate. The storage device606 may be internal or external to the computing device 600. In one ormore embodiments, the storage device 606 is non-volatile, solid-statememory. In other embodiments, the storage device 606 includes read-onlymemory (ROM). Where appropriate, this ROM may be mask programmed ROM,programmable ROM (PROM), erasable PROM (EPROM), electrically erasablePROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or acombination of two or more of these.

The I/O interface 608 allows a user to provide input to, receive outputfrom, and otherwise transfer data to and receive data from computingdevice 600. The I/O interface 608 may include a mouse, a keypad or akeyboard, a touch screen, a camera, an optical scanner, networkinterface, modem, other known I/O devices or a combination of such I/Ointerfaces. The I/O interface 608 may include one or more devices forpresenting output to a user, including, but not limited to, a graphicsengine, a display (e.g., a display screen), one or more output drivers(e.g., display drivers), one or more audio speakers, and one or moreaudio drivers. In certain embodiments, the I/O interface 608 isconfigured to provide graphical data to a display for presentation to auser. The graphical data may be representative of one or more graphicaluser interfaces and/or any other graphical content as may serve aparticular implementation.

The communication interface 610 can include hardware, software, or both.In any event, the communication interface 610 can provide one or moreinterfaces for communication (such as, for example, packet-basedcommunication) between the computing device 600 and one or more othercomputing devices or networks. As an example and not by way oflimitation, the communication interface 610 may include a networkinterface controller (NIC) or network adapter for communicating with anEthernet or other wire-based network or a wireless NIC (WNIC) orwireless adapter for communicating with a wireless network, such as aWI-FI.

Additionally or alternatively, the communication interface 610 mayfacilitate communications with an ad hoc network, a personal areanetwork (PAN), a local area network (LAN), a wide area network (WAN), ametropolitan area network (MAN), or one or more portions of the Internetor a combination of two or more of these. One or more portions of one ormore of these networks may be wired or wireless. As an example, thecommunication interface 610 may facilitate communications with awireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FInetwork, a WI-MAX network, a cellular telephone network (such as, forexample, a Global System for Mobile Communications (GSM) network), orother suitable wireless network or a combination thereof.

Additionally, the communication interface 610 may facilitatecommunications various communication protocols. Examples ofcommunication protocols that may be used include, but are not limitedto, data transmission media, communications devices, TransmissionControl Protocol (“TCP”), Internet Protocol (“IP”), File TransferProtocol (“FTP”), Telnet, Hypertext Transfer Protocol (“HTTP”),Hypertext Transfer Protocol Secure (“HTTPS”), Session InitiationProtocol (“SIP”), Simple Object Access Protocol (“SOAP”), ExtensibleMark-up Language (“XML”) and variations thereof, Simple Mail TransferProtocol (“SMTP”), Real-Time Transport Protocol (“RTP”), User DatagramProtocol (“UDP”), Global System for Mobile Communications (“GSM”)technologies, Code Division Multiple Access (“CDMA”) technologies, TimeDivision Multiple Access (“TDMA”) technologies, Short Message Service(“SMS”), Multimedia Message Service (“MMS”), radio frequency (“RF”)signaling technologies, Long Term Evolution (“LTE”) technologies,wireless communication technologies, in-band and out-of-band signalingtechnologies, and other suitable communications networks andtechnologies.

The communication infrastructure 612 may include hardware, software, orboth that couples components of the computing device 600 to each other.As an example and not by way of limitation, the communicationinfrastructure 612 may include an Accelerated Graphics Port (AGP) orother graphics bus, an Enhanced Industry Standard Architecture (EISA)bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, anIndustry Standard Architecture (ISA) bus, an INFINIBAND interconnect, alow-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture(MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express(PCIe) bus, a serial advanced technology attachment (SATA) bus, a VideoElectronics Standards Association local (VLB) bus, or another suitablebus or a combination thereof.

As mentioned above, the augmented reality system 100 can comprise asocial networking system. A social networking system may enable itsusers (such as persons or organizations) to interact with the system andwith each other. The social networking system may, with input from auser, create and store in the social networking system a user profileassociated with the user. The user profile may include demographicinformation, communication-channel information, and information onpersonal interests of the user. The social networking system may also,with input from a user, create and store a record of relationships ofthe user with other users of the social networking system, as well asprovide services (e.g., posts, photo-sharing, event organization,messaging, games, or advertisements) to facilitate social interactionbetween or among users.

The social networking system may store records of users andrelationships between users in a social graph comprising a plurality ofnodes and a plurality of edges connecting the nodes. The nodes maycomprise a plurality of user nodes and a plurality of concept nodes. Auser node of the social graph may correspond to a user of the socialnetworking system. A user may be an individual (human user), an entity(e.g., an enterprise, business, or third party application), or a group(e.g., of individuals or entities). A user node corresponding to a usermay comprise information provided by the user and information gatheredby various systems, including the social networking system.

For example, the user may provide his or her name, profile picture, cityof residence, contact information, birth date, gender, marital status,family status, employment, educational background, preferences,interests, and other demographic information to be included in the usernode. Each user node of the social graph may have a corresponding webpage (typically known as a profile page). In response to a requestincluding a user name, the social networking system can access a usernode corresponding to the user name, and construct a profile pageincluding the name, a profile picture, and other information associatedwith the user. A profile page of a first user may display to a seconduser all or a portion of the first user's information based on one ormore privacy settings by the first user and the relationship between thefirst user and the second user.

A concept node may correspond to a concept of the social networkingsystem. For example, a concept can represent a real-world entity, suchas a movie, a song, a sports team, a celebrity, a group, a restaurant,or a place or a location. An administrative user of a concept nodecorresponding to a concept may create or update the concept node byproviding information of the concept (e.g., by filling out an onlineform), causing the social networking system to associate the informationwith the concept node. For example and without limitation, informationassociated with a concept can include a name or a title, one or moreimages (e.g., an image of cover page of a book), a web site (e.g., anURL address) or contact information (e.g., a phone number, an emailaddress). Each concept node of the social graph may correspond to a webpage. For example, in response to a request including a name, the socialnetworking system can access a concept node corresponding to the name,and construct a web page including the name and other informationassociated with the concept.

An edge between a pair of nodes may represent a relationship between thepair of nodes. For example, an edge between two user nodes can representa friendship between two users. For another example, the socialnetworking system may construct a web page (or a structured document) ofa concept node (e.g., a restaurant, a celebrity), incorporating one ormore selectable option or selectable elements (e.g., “like”, “check in”)in the web page. A user can access the page using a web browser hostedby the user's client device and select a selectable option or selectableelement, causing the client device to transmit to the social networkingsystem a request to create an edge between a user node of the user and aconcept node of the concept, indicating a relationship between the userand the concept (e.g., the user checks in a restaurant, or the user“likes” a celebrity).

As an example, a user may provide (or change) his or her city ofresidence, causing the social networking system to create an edgebetween a user node corresponding to the user and a concept nodecorresponding to the city declared by the user as his or her city ofresidence. In addition, the degree of separation between any two nodesis defined as the minimum number of hops required to traverse the socialgraph from one node to the other. A degree of separation between twonodes can be considered a measure of relatedness between the users orthe concepts represented by the two nodes in the social graph. Forexample, two users having user nodes that are directly connected by anedge (i.e., are first-degree nodes) may be described as “connectedusers” or “friends.” Similarly, two users having user nodes that areconnected only through another user node (i.e., are second-degree nodes)may be described as “friends of friends.”

A social networking system may support a variety of applications, suchas photo sharing, on-line calendars and events, gaming, instantmessaging, and advertising. For example, the social networking systemmay also include media sharing capabilities. Also, the social networkingsystem may allow users to post photographs and other multimedia contentitems to a user's profile page (typically known as “wall posts” or“timeline posts”) or in a photo album, both of which may be accessibleto other users of the social networking system depending upon the user'sconfigured privacy settings. The social networking system may also allowusers to configure events. For example, a first user may configure anevent with attributes including time and date of the event, location ofthe event and other users invited to the event. The invited users mayreceive invitations to the event and respond (such as by accepting theinvitation or declining it). Furthermore, the social networking systemmay allow users to maintain a personal calendar. Similarly to events,the calendar entries may include times, dates, locations and identitiesof other users.

FIG. 7 illustrates an example network environment 700 of a socialnetworking system. Network environment 700 includes a client device 706,a networking system 702, and a third-party system 708 connected to eachother by a network 704. Although FIG. 7 illustrates a particulararrangement of client device 706, networking system 702, third-partysystem 708, and network 704, this disclosure contemplates any suitablearrangement of client device 706, networking system 702, third-partysystem 708, and network 704. As an example and not by way of limitation,two or more of client device 706, networking system 702, and third-partysystem 708 may be connected to each other directly, bypassing network704. As another example, two or more of client device 706, networkingsystem 702, and third-party system 708 may be physically or logicallyco-located with each other in whole or in part. Moreover, although FIG.7 illustrates a particular number of client devices 706, networkingsystems 702, third-party systems 708, and networks 704, this disclosurecontemplates any suitable number of client devices 706, networkingsystems 702, third-party systems 708, and networks 704. As an exampleand not by way of limitation, network environment 700 may includemultiple client device 706, networking systems 702, third-party systems708, and networks 704.

This disclosure contemplates any suitable network 704. As an example andnot by way of limitation, one or more portions of network 704 mayinclude an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), a portion of the Internet, a portion of the Public SwitchedTelephone Network (PSTN), a cellular telephone network, or a combinationof two or more of these. Network 704 may include one or more networks704.

Links may connect client device 706, networking system 702, andthird-party system 708 to communication network 704 or to each other.This disclosure contemplates any suitable links. In particularembodiments, one or more links include one or more wireline (such as forexample Digital Subscriber Line (DSL) or Data Over Cable ServiceInterface Specification (DOC SIS)), wireless (such as for example Wi-Fior Worldwide Interoperability for Microwave Access (WiMAX)), or optical(such as for example Synchronous Optical Network (SONET) or SynchronousDigital Hierarchy (SDH)) links. In particular embodiments, one or morelinks each include an ad hoc network, an intranet, an extranet, a VPN, aLAN, a WLAN, a WAN, a WWAN, a MAN, a portion of the Internet, a portionof the PSTN, a cellular technology-based network, a satellitecommunications technology-based network, another link, or a combinationof two or more such links. Links need not necessarily be the samethroughout network environment 700. One or more first links may differin one or more respects from one or more second links.

In particular embodiments, client device 706 may be an electronic deviceincluding hardware, software, or embedded logic components or acombination of two or more such components and capable of carrying outthe appropriate functionalities implemented or supported by clientdevice 706. As an example and not by way of limitation, a client device706 may include a computer system such as an augmented reality displaydevice, a desktop computer, notebook or laptop computer, netbook, atablet computer, e-book reader, GPS device, camera, personal digitalassistant (PDA), handheld electronic device, cellular telephone,smartphone, other suitable electronic device, or any suitablecombination thereof. This disclosure contemplates any suitable clientdevices 706. A client device 706 may enable a network user at clientdevice 706 to access network 704. A client device 706 may enable itsuser to communicate with other users at other client devices 706.

In particular embodiments, client device 706 may include a web browser,such as MICROSOFT INTERNET EXPLORER, GOOGLE CHROME or MOZILLA FIREFOX,and may have one or more add-ons, plug-ins, or other extensions, such asTOOLBAR or YAHOO TOOLBAR. A user at client device 706 may enter aUniform Resource Locator (URL) or other address directing the webbrowser to a particular server (such as server, or a server associatedwith a third-party system 708), and the web browser may generate a HyperText Transfer Protocol (HTTP) request and communicate the HTTP requestto server. The server may accept the HTTP request and communicate toclient device 706 one or more Hyper Text Markup Language (HTML) filesresponsive to the HTTP request. Client device 706 may render a webpagebased on the HTML files from the server for presentation to the user.This disclosure contemplates any suitable webpage files. As an exampleand not by way of limitation, webpages may render from HTML files,Extensible Hyper Text Markup Language (XHTML) files, or ExtensibleMarkup Language (XML) files, according to particular needs. Such pagesmay also execute scripts such as, for example and without limitation,those written in JAVASCRIPT, JAVA, MICROSOFT SILVERLIGHT, combinationsof markup language and scripts such as AJAX (Asynchronous JAVASCRIPT andXML), and the like. Herein, reference to a webpage encompasses one ormore corresponding webpage files (which a browser may use to render thewebpage) and vice versa, where appropriate.

In particular embodiments, networking system 702 may be anetwork-addressable computing system that can host an online socialnetwork. Networking system 702 may generate, store, receive, and sendsocial-networking data, such as, for example, user-profile data,concept-profile data, social-graph information, or other suitable datarelated to the online social network. Networking system 702 may beaccessed by the other components of network environment 700 eitherdirectly or via network 704. In particular embodiments, networkingsystem 702 may include one or more servers. Each server may be a unitaryserver or a distributed server spanning multiple computers or multipledatacenters. Servers may be of various types, such as, for example andwithout limitation, web server, news server, mail server, messageserver, advertising server, file server, application server, exchangeserver, database server, proxy server, another server suitable forperforming functions or processes described herein, or any combinationthereof. In particular embodiments, each server may include hardware,software, or embedded logic components or a combination of two or moresuch components for carrying out the appropriate functionalitiesimplemented or supported by server. In particular embodiments,networking system 702 may include one or more data stores. Data storesmay be used to store various types of information. In particularembodiments, the information stored in data stores may be organizedaccording to specific data structures. In particular embodiments, eachdata store may be a relational, columnar, correlation, or other suitabledatabase. Although this disclosure describes or illustrates particulartypes of databases, this disclosure contemplates any suitable types ofdatabases. Particular embodiments may provide interfaces that enable aclient device 706, a networking system 702, or a third-party system 708to manage, retrieve, modify, add, or delete, the information stored indata store.

In particular embodiments, networking system 702 may store one or moresocial graphs in one or more data stores. In particular embodiments, asocial graph may include multiple nodes—which may include multiple usernodes (each corresponding to a particular user) or multiple conceptnodes (each corresponding to a particular concept)—and multiple edgesconnecting the nodes. Networking system 702 may provide users of theonline social network the ability to communicate and interact with otherusers. In particular embodiments, users may join the online socialnetwork via networking system 702 and then add connections (e.g.,relationships) to a number of other users of networking system 702 thatthey want to be connected to. Herein, the term “friend” may refer to anyother user of networking system 702 with whom a user has formed aconnection, association, or relationship via networking system 702.

In particular embodiments, networking system 702 may provide users withthe ability to take actions on various types of items or objects,supported by networking system 702. As an example and not by way oflimitation, the items and objects may include groups or social networksto which users of networking system 702 may belong, events or calendarentries in which a user might be interested, computer-based applicationsthat a user may use, transactions that allow users to buy or sell itemsvia the service, interactions with advertisements that a user mayperform, or other suitable items or objects. A user may interact withanything that is capable of being represented in networking system 702or by an external system of third-party system 708, which is separatefrom networking system 702 and coupled to networking system 702 via anetwork 704.

In particular embodiments, networking system 702 may be capable oflinking a variety of entities. As an example and not by way oflimitation, networking system 702 may enable users to interact with eachother as well as receive content from third-party systems 708 or otherentities, or to allow users to interact with these entities through anapplication programming interfaces (API) or other communicationchannels.

In particular embodiments, a third-party system 708 may include one ormore types of servers, one or more data stores, one or more interfaces,including but not limited to APIs, one or more web services, one or morecontent sources, one or more networks, or any other suitable components,e.g., that servers may communicate with. A third-party system 708 may beoperated by a different entity from an entity operating networkingsystem 702. In particular embodiments, however, networking system 702and third-party systems 708 may operate in conjunction with each otherto provide social-networking services to users of networking system 702or third-party systems 708. In this sense, networking system 702 mayprovide a platform, or backbone, which other systems, such asthird-party systems 708, may use to provide social-networking servicesand functionality to users across the Internet.

In particular embodiments, a third-party system 708 may include athird-party content object provider. A third-party content objectprovider may include one or more sources of content objects, which maybe communicated to a client device 706. As an example and not by way oflimitation, content objects may include information regarding things oractivities of interest to the user, such as, for example, movie showtimes, movie reviews, restaurant reviews, restaurant menus, productinformation and reviews, or other suitable information. As anotherexample and not by way of limitation, content objects may includeincentive content objects, such as coupons, discount tickets, giftcertificates, or other suitable incentive objects.

In particular embodiments, networking system 702 also includesuser-generated content objects, which may enhance a user's interactionswith networking system 702. User-generated content may include anythinga user can add, upload, send, or “post” to networking system 702. As anexample and not by way of limitation, a user communicates posts tonetworking system 702 from a client device 706. Posts may include datasuch as status updates or other textual data, location information,photos, videos, links, music or other similar data or media. Content mayalso be added to networking system 702 by a third-party through a“communication channel,” such as a newsfeed or stream.

In particular embodiments, networking system 702 may include a varietyof servers, sub-systems, programs, modules, logs, and data stores. Inparticular embodiments, networking system 702 may include one or more ofthe following: a web server, action logger, API-request server,relevance-and-ranking engine, content-object classifier, notificationcontroller, action log, third-party-content-object-exposure log,inference module, authorization/privacy server, search module,advertisement-targeting module, user-interface module, user-profilestore, connection store, third-party content store, or location store.Networking system 702 may also include suitable components such asnetwork interfaces, security mechanisms, load balancers, failoverservers, management-and-network-operations consoles, other suitablecomponents, or any suitable combination thereof. In particularembodiments, networking system 702 may include one or more user-profilestores for storing user profiles. A user profile may include, forexample, biographic information, demographic information, behavioralinformation, social information, or other types of descriptiveinformation, such as work experience, educational history, hobbies orpreferences, interests, affinities, or location. Interest informationmay include interests related to one or more categories. Categories maybe general or specific. As an example and not by way of limitation, if auser “likes” an article about a brand of shoes the category may be thebrand, or the general category of “shoes” or “clothing.” A connectionstore may be used for storing connection information about users. Theconnection information may indicate users who have similar or commonwork experience, group memberships, hobbies, educational history, or arein any way related or share common attributes. The connectioninformation may also include user-defined connections between differentusers and content (both internal and external). A web server may be usedfor linking networking system 702 to one or more client devices 706 orone or more third-party system 708 via network 704. The web server mayinclude a mail server or other messaging functionality for receiving androuting messages between networking system 702 and one or more clientdevices 706. An API-request server may allow a third-party system 708 toaccess information from networking system 702 by calling one or moreAPIs. An action logger may be used to receive communications from a webserver about a user's actions on or off networking system 702. Inconjunction with the action log, a third-party-content-object log may bemaintained of user exposures to third-party-content objects. Anotification controller may provide information regarding contentobjects to a client device 706. Information may be pushed to a clientdevice 706 as notifications, or information may be pulled from clientdevice 706 responsive to a request received from client device 706.Authorization servers may be used to enforce one or more privacysettings of the users of networking system 702. A privacy setting of auser determines how particular information associated with a user can beshared. The authorization server may allow users to opt in to or opt outof having their actions logged by networking system 702 or shared withother systems (e.g., third-party system 708), such as, for example, bysetting appropriate privacy settings. Third-party-content-object storesmay be used to store content objects received from third parties, suchas a third-party system 708. Location stores may be used for storinglocation information received from client devices 706 associated withusers. Advertisement-pricing modules may combine social information, thecurrent time, location information, or other suitable information toprovide relevant advertisements, in the form of notifications, to auser.

FIG. 8 illustrates example social graph 800. In particular embodiments,networking system 702 may store one or more social graphs 800 in one ormore data stores. In particular embodiments, social graph 800 mayinclude multiple nodes—which may include multiple user nodes 802 ormultiple concept nodes 804—and multiple edges 806 connecting the nodes.Example social graph 800 illustrated in FIG. 8 is shown, for didacticpurposes, in a two-dimensional visual map representation. In particularembodiments, a networking system 702, client device 706, or third-partysystem 708 may access social graph 800 and related social-graphinformation for suitable applications. The nodes and edges of socialgraph 800 may be stored as data objects, for example, in a data store(such as a social-graph database). Such a data store may include one ormore searchable or query able indexes of nodes or edges of social graph800.

In particular embodiments, a user node 802 may correspond to a user ofnetworking system 702. As an example and not by way of limitation, auser may be an individual (human user), an entity (e.g., an enterprise,business, or third-party application), or a group (e.g., of individualsor entities) that interacts or communicates with or over networkingsystem 702. In particular embodiments, when a user registers for anaccount with networking system 702, networking system 702 may create auser node 802 corresponding to the user, and store the user node 802 inone or more data stores. Users and user nodes 802 described herein may,where appropriate, refer to registered users and user nodes 802associated with registered users. In addition or as an alternative,users and user nodes 802 described herein may, where appropriate, referto users that have not registered with networking system 702. Inparticular embodiments, a user node 802 may be associated withinformation provided by a user or information gathered by varioussystems, including networking system 702. As an example and not by wayof limitation, a user may provide his or her name, profile picture,contact information, birth date, sex, marital status, family status,employment, education background, preferences, interests, or otherdemographic information. In particular embodiments, a user node 802 maybe associated with one or more data objects corresponding to informationassociated with a user. In particular embodiments, a user node 802 maycorrespond to one or more webpages.

In particular embodiments, a concept node 804 may correspond to aconcept. As an example and not by way of limitation, a concept maycorrespond to a place (such as, for example, a movie theater,restaurant, landmark, or city); a website (such as, for example, awebsite associated with networking system 702 or a third-party websiteassociated with a web-application server); an entity (such as, forexample, a person, business, group, sports team, or celebrity); aresource (such as, for example, an audio file, video file, digitalphoto, text file, structured document, or application) which may belocated within networking system 702 or on an external server, such as aweb-application server; real or intellectual property (such as, forexample, a sculpture, painting, movie, game, song, idea, photograph, orwritten work); a game; an activity; an idea or theory; another suitableconcept; or two or more such concepts. A concept node 804 may beassociated with information of a concept provided by a user orinformation gathered by various systems, including networking system702. As an example and not by way of limitation, information of aconcept may include a name or a title; one or more images (e.g., animage of the cover page of a book); a location (e.g., an address or ageographical location); a website (which may be associated with a URL);contact information (e.g., a phone number or an email address); othersuitable concept information; or any suitable combination of suchinformation. In particular embodiments, a concept node 804 may beassociated with one or more data objects corresponding to informationassociated with concept node 804. In particular embodiments, a conceptnode 804 may correspond to one or more webpages.

In particular embodiments, a node in social graph 800 may represent orbe represented by a webpage (which may be referred to as a “profilepage”). Profile pages may be hosted by or accessible to networkingsystem 702. Profile pages may also be hosted on third-party websitesassociated with a third-party system 708. As an example and not by wayof limitation, a profile page corresponding to a particular externalwebpage may be the particular external webpage and the profile page maycorrespond to a particular concept node 804. Profile pages may beviewable by all or a selected subset of other users. As an example andnot by way of limitation, a user node 802 may have a correspondinguser-profile page in which the corresponding user may add content, makedeclarations, or otherwise express himself or herself. As anotherexample and not by way of limitation, a concept node 804 may have acorresponding concept-profile page in which one or more users may addcontent, make declarations, or express themselves, particularly inrelation to the concept corresponding to concept node 804.

In particular embodiments, a concept node 804 may represent athird-party webpage or resource hosted by a third-party system 708. Thethird-party webpage or resource may include, among other elements,content, a selectable or other icon, or other inter-actable object(which may be implemented, for example, in JavaScript, AJAX, or PHPcodes) representing an action or activity. As an example and not by wayof limitation, a third-party webpage may include a selectable icon suchas “like,” “check in,” “eat,” “recommend,” or another suitable action oractivity. A user viewing the third-party webpage may perform an actionby selecting one of the icons (e.g., “eat”), causing a client device 706to send to networking system 702 a message indicating the user's action.In response to the message, networking system 702 may create an edge(e.g., an “eat” edge) between a user node 802 corresponding to the userand a concept node 804 corresponding to the third-party webpage orresource and store edge 806 in one or more data stores.

In particular embodiments, a pair of nodes in social graph 800 may beconnected to each other by one or more edges 806. An edge 806 connectinga pair of nodes may represent a relationship between the pair of nodes.In particular embodiments, an edge 806 may include or represent one ormore data objects or attributes corresponding to the relationshipbetween a pair of nodes. As an example and not by way of limitation, afirst user may indicate that a second user is a “friend” of the firstuser. In response to this indication, networking system 702 may send a“friend request” to the second user. If the second user confirms the“friend request,” networking system 702 may create an edge 806connecting the first user's user node 802 to the second user's user node802 in social graph 800 and store edge 806 as social-graph informationin one or more of data stores. In the example of FIG. 8, social graph800 includes an edge 806 indicating a friend relation between user nodes802 of user “A” and user “B” and an edge indicating a friend relationbetween user nodes 802 of user “C” and user “B.” Although thisdisclosure describes or illustrates particular edges 806 with particularattributes connecting particular user nodes 802, this disclosurecontemplates any suitable edges 806 with any suitable attributesconnecting user nodes 802. As an example and not by way of limitation,an edge 806 may represent a friendship, family relationship, business oremployment relationship, fan relationship, follower relationship,visitor relationship, subscriber relationship, superior/subordinaterelationship, reciprocal relationship, non-reciprocal relationship,another suitable type of relationship, or two or more suchrelationships. Moreover, although this disclosure generally describesnodes as being connected, this disclosure also describes users orconcepts as being connected. Herein, references to users or conceptsbeing connected may, where appropriate, refer to the nodes correspondingto those users or concepts being connected in social graph 800 by one ormore edges 806.

In particular embodiments, an edge 806 between a user node 802 and aconcept node 804 may represent a particular action or activity performedby a user associated with user node 802 toward a concept associated witha concept node 804. As an example and not by way of limitation, asillustrated in FIG. 8, a user may “like,” “attended,” “played,”“listened,” “cooked,” “worked at,” or “watched” a concept, each of whichmay correspond to a edge type or subtype. A concept-profile pagecorresponding to a concept node 804 may include, for example, aselectable “check in” icon (such as, for example, a clickable “check in”icon) or a selectable “add to favorites” icon. Similarly, after a userclicks these icons, networking system 702 may create a “favorite” edgeor a “check in” edge in response to a user's action corresponding to arespective action. As another example and not by way of limitation, auser (user “C”) may listen to a particular song (“Ramble On”) using aparticular application (SPOTIFY, which is an online music application).In this case, networking system 702 may create a “listened” edge 806 anda “used” edge (as illustrated in FIG. 8) between user nodes 802corresponding to the user and concept nodes 804 corresponding to thesong and application to indicate that the user listened to the song andused the application. Moreover, networking system 702 may create a“played” edge 806 (as illustrated in FIG. 8) between concept nodes 804corresponding to the song and the application to indicate that theparticular song was played by the particular application. In this case,“played” edge 806 corresponds to an action performed by an externalapplication (SPOTIFY) on an external audio file (the song “Imagine”).Although this disclosure describes particular edges 806 with particularattributes connecting user nodes 802 and concept nodes 804, thisdisclosure contemplates any suitable edges 806 with any suitableattributes connecting user nodes 802 and concept nodes 804. Moreover,although this disclosure describes edges between a user node 802 and aconcept node 804 representing a single relationship, this disclosurecontemplates edges between a user node 802 and a concept node 804representing one or more relationships. As an example and not by way oflimitation, an edge 806 may represent both that a user likes and hasused at a particular concept. Alternatively, another edge 806 mayrepresent each type of relationship (or multiples of a singlerelationship) between a user node 802 and a concept node 804 (asillustrated in FIG. 8 between user node 802 for user “E” and conceptnode 804 for “SPOTIFY”).

In particular embodiments, networking system 702 may create an edge 806between a user node 802 and a concept node 804 in social graph 800. Asan example and not by way of limitation, a user viewing aconcept-profile page (such as, for example, by using a web browser or aspecial-purpose application hosted by the user's client device 706) mayindicate that he or she likes the concept represented by the conceptnode 804 by clicking or selecting a “Like” icon, which may cause theuser's client device 706 to send to networking system 702 a messageindicating the user's liking of the concept associated with theconcept-profile page. In response to the message, networking system 702may create an edge 806 between user node 802 associated with the userand concept node 804, as illustrated by “like” edge 806 between the userand concept node 804. In particular embodiments, networking system 702may store an edge 806 in one or more data stores. In particularembodiments, an edge 806 may be automatically formed by networkingsystem 702 in response to a particular user action. As an example andnot by way of limitation, if a first user uploads a picture, watches amovie, or listens to a song, an edge 806 may be formed between user node802 corresponding to the first user and concept nodes 804 correspondingto those concepts. Although this disclosure describes forming particularedges 806 in particular manners, this disclosure contemplates formingany suitable edges 806 in any suitable manner.

In particular embodiments, an advertisement may be text (which may beHTML-linked), one or more images (which may be HTML-linked), one or morevideos, audio, one or more ADOBE FLASH files, a suitable combination ofthese, or any other suitable advertisement in any suitable digitalformat presented on one or more webpages, in one or more e-mails, or inconnection with search results requested by a user. In addition or as analternative, an advertisement may be one or more sponsored stories(e.g., a news-feed or ticker item on networking system 702). A sponsoredstory may be a social action by a user (such as “liking” a page,“liking” or commenting on a post on a page, RSVPing to an eventassociated with a page, voting on a question posted on a page, checkingin to a place, using an application or playing a game, or “liking” orsharing a website) that an advertiser promotes, for example, by havingthe social action presented within a pre-determined area of a profilepage of a user or other page, presented with additional informationassociated with the advertiser, bumped up or otherwise highlightedwithin news feeds or tickers of other users, or otherwise promoted. Theadvertiser may pay to have the social action promoted. As an example andnot by way of limitation, advertisements may be included among thesearch results of a search-results page, where sponsored content ispromoted over non-sponsored content.

In particular embodiments, an advertisement may be requested for displaywithin social-networking-system webpages, third-party webpages, or otherpages. An advertisement may be displayed in a dedicated portion of apage, such as in a banner area at the top of the page, in a column atthe side of the page, in a GUI of the page, in a pop-up window, in adrop-down menu, in an input field of the page, over the top of contentof the page, or elsewhere with respect to the page. In addition or as analternative, an advertisement may be displayed within an application. Anadvertisement may be displayed within dedicated pages, requiring theuser to interact with or watch the advertisement before the user mayaccess a page or utilize an application. The user may, for example viewthe advertisement through a web browser.

A user may interact with an advertisement in any suitable manner. Theuser may click or otherwise select the advertisement. By selecting theadvertisement, the user may be directed to (or a browser or otherapplication being used by the user) a page associated with theadvertisement. At the page associated with the advertisement, the usermay take additional actions, such as purchasing a product or serviceassociated with the advertisement, receiving information associated withthe advertisement, or subscribing to a newsletter associated with theadvertisement. An advertisement with audio or video may be played byselecting a component of the advertisement (like a “play button”).Alternatively, by selecting the advertisement, networking system 702 mayexecute or modify a particular action of the user.

An advertisement may also include social-networking-system functionalitythat a user may interact with. As an example and not by way oflimitation, an advertisement may enable a user to “like” or otherwiseendorse the advertisement by selecting an icon or link associated withendorsement. As another example and not by way of limitation, anadvertisement may enable a user to search (e.g., by executing a query)for content related to the advertiser. Similarly, a user may share theadvertisement with another user (e.g., through networking system 702) orRSVP (e.g., through networking system 702) to an event associated withthe advertisement. In addition or as an alternative, an advertisementmay include social-networking-system context directed to the user. As anexample and not by way of limitation, an advertisement may displayinformation about a friend of the user within networking system 702 whohas taken an action associated with the subject matter of theadvertisement.

In particular embodiments, networking system 702 may determine thesocial-graph affinity (which may be referred to herein as “affinity”) ofvarious social-graph entities for each other. Affinity may represent thestrength of a relationship or level of interest between particularobjects associated with the online social network, such as users,concepts, content, actions, advertisements, other objects associatedwith the online social network, or any suitable combination thereof.Affinity may also be determined with respect to objects associated withthird-party systems 708 or other suitable systems. An overall affinityfor a social-graph entity for each user, subject matter, or type ofcontent may be established. The overall affinity may change based oncontinued monitoring of the actions or relationships associated with thesocial-graph entity. Although this disclosure describes determiningparticular affinities in a particular manner, this disclosurecontemplates determining any suitable affinities in any suitable manner.

In particular embodiments, networking system 702 may measure or quantifysocial-graph affinity using an affinity coefficient (which may bereferred to herein as “coefficient”). The coefficient may represent orquantify the strength of a relationship between particular objectsassociated with the online social network. The coefficient may alsorepresent a probability or function that measures a predictedprobability that a user will perform a particular action based on theuser's interest in the action. In this way, a user's future actions maybe predicted based on the user's prior actions, where the coefficientmay be calculated at least in part a the history of the user's actions.Coefficients may be used to predict any number of actions, which may bewithin or outside of the online social network. As an example and not byway of limitation, these actions may include various types ofcommunications, such as sending messages, posting content, or commentingon content; various types of a observation actions, such as accessing orviewing profile pages, media, or other suitable content; various typesof coincidence information about two or more social-graph entities, suchas being in the same group, tagged in the same photograph, checked-in atthe same location, or attending the same event; or other suitableactions. Although this disclosure describes measuring affinity in aparticular manner, this disclosure contemplates measuring affinity inany suitable manner.

In particular embodiments, networking system 702 may use a variety offactors to calculate a coefficient. These factors may include, forexample, user actions, types of relationships between objects, locationinformation, other suitable factors, or any combination thereof. Inparticular embodiments, different factors may be weighted differentlywhen calculating the coefficient. The weights for each factor may bestatic or the weights may change according to, for example, the user,the type of relationship, the type of action, the user's location, andso forth. Ratings for the factors may be combined according to theirweights to determine an overall coefficient for the user. As an exampleand not by way of limitation, particular user actions may be assignedboth a rating and a weight while a relationship associated with theparticular user action is assigned a rating and a correlating weight(e.g., so the weights total 100%). To calculate the coefficient of auser towards a particular object, the rating assigned to the user'sactions may comprise, for example, 60% of the overall coefficient, whilethe relationship between the user and the object may comprise 40% of theoverall coefficient. In particular embodiments, the networking system702 may consider a variety of variables when determining weights forvarious factors used to calculate a coefficient, such as, for example,the time since information was accessed, decay factors, frequency ofaccess, relationship to information or relationship to the object aboutwhich information was accessed, relationship to social-graph entitiesconnected to the object, short- or long-term averages of user actions,user feedback, other suitable variables, or any combination thereof. Asan example and not by way of limitation, a coefficient may include adecay factor that causes the strength of the signal provided byparticular actions to decay with time, such that more recent actions aremore relevant when calculating the coefficient. The ratings and weightsmay be continuously updated based on continued tracking of the actionsupon which the coefficient is based. Any type of process or algorithmmay be employed for assigning, combining, averaging, and so forth theratings for each factor and the weights assigned to the factors. Inparticular embodiments, networking system 702 may determine coefficientsusing machine-learning algorithms trained on historical actions and pastuser responses, or data farmed from users by exposing them to variousoptions and measuring responses. Although this disclosure describescalculating coefficients in a particular manner, this disclosurecontemplates calculating coefficients in any suitable manner.

In particular embodiments, networking system 702 may calculate acoefficient based on a user's actions. Networking system 702 may monitorsuch actions on the online social network, on a third-party system 708,on other suitable systems, or any combination thereof. Any suitable typeof user actions may be tracked or monitored. Typical user actionsinclude viewing profile pages, creating or posting content, interactingwith content, joining groups, listing and confirming attendance atevents, checking-in at locations, liking particular pages, creatingpages, and performing other tasks that facilitate social action. Inparticular embodiments, networking system 702 may calculate acoefficient based on the user's actions with particular types ofcontent. The content may be associated with the online social network, athird-party system 708, or another suitable system. The content mayinclude users, profile pages, posts, news stories, headlines, instantmessages, chat room conversations, emails, advertisements, pictures,video, music, other suitable objects, or any combination thereof.Networking system 702 may analyze a user's actions to determine whetherone or more of the actions indicate an affinity for subject matter,content, other users, and so forth. As an example and not by way oflimitation, if a user may make frequently posts content related to“coffee” or variants thereof, networking system 702 may determine theuser has a high coefficient with respect to the concept “coffee”.Particular actions or types of actions may be assigned a higher weightand/or rating than other actions, which may affect the overallcalculated coefficient. As an example and not by way of limitation, if afirst user emails a second user, the weight or the rating for the actionmay be higher than if the first user simply views the user-profile pagefor the second user.

In particular embodiments, networking system 702 may calculate acoefficient based on the type of relationship between particularobjects. Referencing the social graph 800, networking system 702 mayanalyze the number and/or type of edges 806 connecting particular usernodes 802 and concept nodes 804 when calculating a coefficient. As anexample and not by way of limitation, user nodes 802 that are connectedby a spouse-type edge (representing that the two users are married) maybe assigned a higher coefficient than a user node 802 that are connectedby a friend-type edge. In other words, depending upon the weightsassigned to the actions and relationships for the particular user, theoverall affinity may be determined to be higher for content about theuser's spouse than for content about the user's friend. In particularembodiments, the relationships a user has with another object may affectthe weights and/or the ratings of the user's actions with respect tocalculating the coefficient for that object. As an example and not byway of limitation, if a user is tagged in first photo, but merely likesa second photo, networking system 702 may determine that the user has ahigher coefficient with respect to the first photo than the second photobecause having a tagged-in-type relationship with content may beassigned a higher weight and/or rating than having a like-typerelationship with content. In particular embodiments, networking system702 may calculate a coefficient for a first user based on therelationship one or more second users have with a particular object. Inother words, the connections and coefficients other users have with anobject may affect the first user's coefficient for the object. As anexample and not by way of limitation, if a first user is connected to orhas a high coefficient for one or more second users, and those secondusers are connected to or have a high coefficient for a particularobject, networking system 702 may determine that the first user shouldalso have a relatively high coefficient for the particular object. Inparticular embodiments, the coefficient may be based on the degree ofseparation between particular objects. The lower coefficient mayrepresent the decreasing likelihood that the first user will share aninterest in content objects of the user that is indirectly connected tothe first user in the social graph 800. As an example and not by way oflimitation, social-graph entities that are closer in the social graph800 (i.e., fewer degrees of separation) may have a higher coefficientthan entities that are further apart in the social graph 800.

In particular embodiments, networking system 702 may calculate acoefficient based on location information. Objects that aregeographically closer to each other may be considered to be morerelated, or of more interest, to each other than more distant objects.In particular embodiments, the coefficient of a user towards aparticular object may be based on the proximity of the object's locationto a current location associated with the user (or the location of aclient device 706 of the user). A first user may be more interested inother users or concepts that are closer to the first user. As an exampleand not by way of limitation, if a user is one mile from an airport andtwo miles from a gas station, networking system 702 may determine thatthe user has a higher coefficient for the airport than the gas stationbased on the proximity of the airport to the user.

In particular embodiments, networking system 702 may perform particularactions with respect to a user based on coefficient information.Coefficients may be used to predict whether a user will perform aparticular action based on the user's interest in the action. Acoefficient may be used when generating or presenting any type ofobjects to a user, such as advertisements, search results, news stories,media, messages, notifications, or other suitable objects. Thecoefficient may also be utilized to rank and order such objects, asappropriate. In this way, networking system 702 may provide informationthat is relevant to user's interests and current circumstances,increasing the likelihood that they will find such information ofinterest. In particular embodiments, networking system 702 may generatecontent based on coefficient information. Content objects may beprovided or selected based on coefficients specific to a user. As anexample and not by way of limitation, the coefficient may be used togenerate media for the user, where the user may be presented with mediafor which the user has a high overall coefficient with respect to themedia object. As another example and not by way of limitation, thecoefficient may be used to generate advertisements for the user, wherethe user may be presented with advertisements for which the user has ahigh overall coefficient with respect to the advertised object. Inparticular embodiments, networking system 702 may generate searchresults based on coefficient information. Search results for aparticular user may be scored or ranked based on the coefficientassociated with the search results with respect to the querying user. Asan example and not by way of limitation, search results corresponding toobjects with higher coefficients may be ranked higher on asearch-results page than results corresponding to objects having lowercoefficients.

In particular embodiments, networking system 702 may calculate acoefficient in response to a request for a coefficient from a particularsystem or process. To predict the likely actions a user may take (or maybe the subject of) in a given situation, any process may request acalculated coefficient for a user. The request may also include a set ofweights to use for various factors used to calculate the coefficient.This request may come from a process running on the online socialnetwork, from a third-party system 708 (e.g., via an API or othercommunication channel), or from another suitable system. In response tothe request, networking system 702 may calculate the coefficient (oraccess the coefficient information if it has previously been calculatedand stored). In particular embodiments, networking system 702 maymeasure an affinity with respect to a particular process. Differentprocesses (both internal and external to the online social network) mayrequest a coefficient for a particular object or set of objects.Networking system 702 may provide a measure of affinity that is relevantto the particular process that requested the measure of affinity. Inthis way, each process receives a measure of affinity that is tailoredfor the different context in which the process will use the measure ofaffinity.

In connection with social-graph affinity and affinity coefficients,particular embodiments may utilize one or more systems, components,elements, functions, methods, operations, or steps disclosed in U.S.patent application Ser. No. 11/503,093, filed 11 Aug. 2006, U.S. patentapplication Ser. No. 12/977,027, filed 22 Dec. 2010, U.S. patentapplication Ser. No. 12/978,265, filed 23 Dec. 2010, and U.S. patentapplication Ser. No. 13/632,869, field 1 Oct. 2012, each of which isincorporated by reference.

In particular embodiments, one or more of the content objects of theonline social network may be associated with a privacy setting. Theprivacy settings (or “access settings”) for an object may be stored inany suitable manner, such as, for example, in association with theobject, in an index on an authorization server, in another suitablemanner, or any combination thereof. A privacy setting of an object mayspecify how the object (or particular information associated with anobject) can be accessed (e.g., viewed or shared) using the online socialnetwork. Where the privacy settings for an object allow a particularuser to access that object, the object may be described as being“visible” with respect to that user. As an example and not by way oflimitation, a user of the online social network may specify privacysettings for a user-profile page identify a set of users that may accessthe work experience information on the user-profile page, thus excludingother users from accessing the information. In particular embodiments,the privacy settings may specify a “blocked list” of users that shouldnot be allowed to access certain information associated with the object.In other words, the blocked list may specify one or more users orentities for which an object is not visible. As an example and not byway of limitation, a user may specify a set of users that may not accessphotos albums associated with the user, thus excluding those users fromaccessing the photo albums (while also possibly allowing certain usersnot within the set of users to access the photo albums). In particularembodiments, privacy settings may be associated with particularsocial-graph elements. Privacy settings of a social-graph element, suchas a node or an edge, may specify how the social-graph element,information associated with the social-graph element, or content objectsassociated with the social-graph element can be accessed using theonline social network. As an example and not by way of limitation, aparticular concept node 804 corresponding to a particular photo may havea privacy setting specifying that the photo may only be accessed byusers tagged in the photo and their friends. In particular embodiments,privacy settings may allow users to opt in or opt out of having theiractions logged by networking system 702 or shared with other systems(e.g., third-party system 708). In particular embodiments, the privacysettings associated with an object may specify any suitable granularityof permitted access or denial of access. As an example and not by way oflimitation, access or denial of access may be specified for particularusers (e.g., only me, my roommates, and my boss), users within aparticular degrees-of-separation (e.g., friends, or friends-of-friends),user groups (e.g., the gaming club, my family), user networks (e.g.,employees of particular employers, students or alumni of particularuniversity), all users (“public”), no users (“private”), users ofthird-party systems 708, particular applications (e.g., third-partyapplications, external websites), other suitable users or entities, orany combination thereof. Although this disclosure describes usingparticular privacy settings in a particular manner, this disclosurecontemplates using any suitable privacy settings in any suitable manner.

In particular embodiments, one or more servers may beauthorization/privacy servers for enforcing privacy settings. Inresponse to a request from a user (or other entity) for a particularobject stored in a data store, networking system 702 may send a requestto the data store for the object. The request may identify the userassociated with the request and may only be sent to the user (or aclient device 706 of the user) if the authorization server determinesthat the user is authorized to access the object based on the privacysettings associated with the object. If the requesting user is notauthorized to access the object, the authorization server may preventthe requested object from being retrieved from the data store, or mayprevent the requested object from be sent to the user. In the searchquery context, an object may only be generated as a search result if thequerying user is authorized to access the object. In other words, theobject must have a visibility that is visible to the querying user. Ifthe object has a visibility that is not visible to the user, the objectmay be excluded from the search results. Although this disclosuredescribes enforcing privacy settings in a particular manner, thisdisclosure contemplates enforcing privacy settings in any suitablemanner.

The foregoing specification is described with reference to specificexemplary embodiments thereof. Various embodiments and aspects of thedisclosure are described with reference to details discussed herein, andthe accompanying drawings illustrate the various embodiments. Thedescription above and drawings are illustrative and are not to beconstrued as limiting. Numerous specific details are described toprovide a thorough understanding of various embodiments.

The additional or alternative embodiments may be embodied in otherspecific forms without departing from its spirit or essentialcharacteristics. The described embodiments are to be considered in allrespects only as illustrative and not restrictive. The scope of theinvention is, therefore, indicated by the appended claims rather than bythe foregoing description. All changes that come within the meaning andrange of equivalency of the claims are to be embraced within theirscope.

1. A method comprising: receiving one or more images captured by anaugmented reality display device associated with a networking systemuser; determining a location of the augmented reality display deviceassociated with the networking system user; determining an identity of anetworking system co-user shown in the one or more images; generating anaugmented reality overlay for the identified networking system co-userby: determining a score for each of a plurality of augmented realityelements based on the location of the augmented reality display device,performing a 3D scan of one of the one or more images to determine the3-dimensional shape of the networking system co-user in one of the oneor more images, generating a graphical mesh based on a point cloud ofdata from the networking system co-user's 3-dimensional shape, and basedon the determined scores for each of the plurality of augmented realityelements, applying one or more of the plurality of augmented realityelements to the graphical mesh; and providing the generated augmentedreality overlay for display by the augmented reality display deviceanchored on a view of the networking system co-user.
 2. The method asrecited in claim 1, wherein determining an identity of the networkingsystem co-user comprises utilizing, in association with the one or moreimages, one or more of facial recognition techniques, image matching, oreye scanning.
 3. The method as recited in claim 1, further comprisingcalculating a networking system relationship coefficient associated withthe relationship between the networking system user and the networkingsystem co-user; and wherein determining a score for each of a pluralityof augmented reality elements is further based on the networking systemrelationship coefficient.
 4. The method as recited in claim 3, whereincalculating the networking system relationship coefficient comprisescalculating the networking system relationship coefficient based onnetworking system activity information common between the networkingsystem user and the networking system co-user.
 5. The method as recitedin claim 1, wherein generating the augmented reality overlay for theidentified networking system co-user is further based on a settingassociated with the augmented reality display device.
 6. The method asrecited in claim 5, wherein determining the location of the augmentedreality display device comprises utilizing one or more of GPSinformation, WiFi information, networking system information, or imageanalysis information from the one or more images.
 7. The method asrecited in claim 6, further comprising determining the settingassociated with the augmented reality display device based on ananalysis of the one or more images.
 8. The method as recited in claim 4,wherein generating the augmented reality overlay for the identifiednetworking system co-user further comprises: identifying the pluralityof augmented reality elements by identifying augmented reality elementsthat are appropriate to the location from a repository of augmentedreality elements; further scoring the identified one or more augmentedreality elements based on predefined rules associated with the co-user;weighting the scores for each of the identified one or more augmentedreality elements based on the networking system relationshipcoefficient, and on networking system information associated with theco-user; identifying a plurality of the scored augmented realityelements with weighted scores above a threshold amount; and applying theplurality of scored augmented reality elements to the graphical mesh. 9.The method as recited in claim 8, wherein applying the plurality ofscored augmented reality elements to the graphical mesh comprisesattaching the plurality of scored augmented reality elements based on aset of reality-based rules that ensure an appearance of the co-user isrealistic after the augmented reality overlay is displayed on theaugmented reality display device.
 10. A system comprising: at least oneprocessor; and at least one non-transitory computer-readable storagemedium storing instructions thereon that, when executed by the at leastone processor, cause the system to: receive one or more images capturedby an augmented reality display device associated with a networkingsystem user; determine a location of the augmented reality displaydevice associated with the networking system user; determine an identityof a networking system co-user shown in the one or more images; generatean augmented reality overlay for the identified networking systemco-user by: determining a score for each of a plurality of augmentedreality elements based on the location of the augmented reality displaydevice, performing a 3D scan of one of the one or more images todetermine the 3-dimensional shape of the networking system co-user inone of the one or more images, generating a graphical mesh based on apoint cloud of data from the networking system co-user's 3-dimensionalshape, and based on the determined scores for each of the plurality ofaugmented reality elements, applying one or more of the plurality ofaugmented reality elements to the graphical mesh; and provide thegenerated augmented reality overlay for display by the augmented realitydisplay device anchored on a view of the networking system co-user. 11.The system as recited in claim 10, wherein determining an identity ofthe networking system co-user comprises utilizing, in association withthe one or more images, one or more of facial recognition techniques,image matching, or eye scanning.
 12. The system as recited in claim 11,further comprising instructions that, when executed by the at least oneprocessor, cause the system to: calculate the networking systemrelationship coefficient associated with the relationship between thenetworking system user and the networking system co-user; and whereindetermining a score for each of a plurality of augmented realityelements is further based on the networking system relationshipcoefficient.
 13. The system as recited in claim 12, wherein calculatingthe networking system relationship coefficient comprises calculating thenetworking system relationship coefficient based on networking systemactivity information common between the networking system user and thenetworking system co-user.
 14. The system as recited in claim 13,wherein generating the augmented reality overlay for the identifiednetworking system co-user is further based on a setting associated withthe augmented reality display device.
 15. The system as recited in claim14, further comprising instructions that, when executed by the at leastone processor, cause the system to determine the location of theaugmented reality display device by utilizing one or more of GPSinformation, WiFi information, networking system information, and imageanalysis information from the one or more images.
 16. The system asrecited in claim 15, further comprising instructions that, when executedby the at least one processor, cause the system to determine the settingassociated with the augmented reality display device based on ananalysis of the one or more images.
 17. The system as recited in claim16, wherein generating the augmented reality overlay for the identifiednetworking system co-user further comprises: identifying the pluralityof augmented reality elements by identifying augmented reality elementsthat are appropriate to the location from a repository of augmentedreality elements; further scoring the identified one or more augmentedreality elements based on predefined rules associated with thenetworking system co-user; weighting the scores for each of theidentified one or more augmented reality elements based on thenetworking system relationship coefficient, and on networking systeminformation associated with the networking system co-user; identifying aplurality of the scored augmented reality elements with weighted scoresabove a threshold amount; and applying the plurality of scored augmentedreality elements to the graphical mesh.
 18. The system as recited inclaim 17, wherein applying the plurality of scored augmented realityelements to the graphical mesh comprises attaching the plurality ofscored augmented reality elements based on a set of reality-based rulesthat ensure an appearance of the co-user is realistic after theaugmented reality overlay is displayed on the augmented reality displaydevice.
 19. A non-transitory computer-readable medium storinginstructions thereon that, when executed by at least one processor,cause a computer system: receive one or more images captured by anaugmented reality display device associated with a networking systemuser; determine a location of the augmented reality display deviceassociated with the networking system user; determine an identity of anetworking system co-user shown in the one or more images; generate anaugmented reality overlay for the identified networking system co-userby: determining a score for each of a plurality of augmented realityelements based on the location of the augmented reality display device,performing a 3D scan of one of the one or more images to determine the3-dimensional shape of the networking system co-user in one of the oneor more images, generating a graphical mesh based on a point cloud ofdata from the networking system co-user's 3-dimensional shape, and basedon the determined scores for each of the plurality of augmented realityelements, applying one or more of the plurality of augmented realityelements to the graphical mesh; and provide the generated augmentedreality overlay for display on the augmented reality display deviceanchored on a view of the networking system co-user.
 20. Thenon-transitory computer-readable medium as recited in claim 19, whereingenerating the augmented reality overlay for the identified networkingsystem co-user further comprises: identifying the plurality of augmentedreality elements by identifying augmented reality elements that areappropriate to the location associated with the augmented realitydisplay device from a repository of augmented reality elements; furtherscoring the identified one or more augmented reality elements based onpredefined rules associated with the networking system co-user;weighting the scores for each of the identified one or more augmentedreality elements based on a networking system relationship coefficientrepresentative of a networking system relationship between thenetworking system user and the networking system co-user, and onnetworking system information associated with the networking systemco-user; identifying a plurality of the scored augmented realityelements with weighted scores above a threshold amount; and applying theplurality of scored augmented reality elements to the graphical mesh.